FreeJ is a video mixer: an instrument for realtime video manipulation used in the fields of dance theater, veejaying, medical visualization and TV. Its development started in 2001 and continued actively for many years.
What we need to do
We need to interact with multiple layers of video, filtered by effect chains and then mixed together.
We want to write scripts to control video mixes with keyboards, midi signals, OSC messages, wiimotes, video mouse and joysticks; manipulating images, movies, live cameras, particle generators, text scrollers, flash animations and more.
We show the resulting video mix on multiple and remote screens, encode it into a movie and stream it live to the internet.
How FreeJ does it
FreeJ is a commandline application on GNU/Linux, a graphical application on Apple/OSX, a C++ library offering an API for a multimedia framework that relies on different native functions on the operating systems it is ported and, at last, bindings to languages like Python and Ruby (using Swig, more can be implemented as needed.
The code is fairly documented and usable in C++, with full bindings to python, while parsing scripts is done using Spidermonkey, the Mozilla interpreter.
FreeJ has started being developed on a dual-core CPU already in 2001 and has grown with emphasis on multi-threading to run efficiently on modern multi-core computers.
With FreeJ we can overlay, mask, transform and filter multiple layers on the screen. There is no limit to the number of layers that can be mixed. Each layer can be video taken from different sources: movie files, webcams, tv cards, images, rendered text,flash animations and more: even particle generators are there, and effects from the frei0r plugin collection.
Stream online video
long live pirate TV! scream your video to the masses! 😉
FreeJ can produce an Ogg/Theora/Vorbis stream for broadcast on Icecast servers, mixing all the video and grabbing the audio from soundcard. The resulting video can be played on any computer connected via internet (list of compatible players).
Instructions on how to stream with FreeJ.
Here are the recordings of a streaming veejay session by Kysucix, screened live at the Linux Audio Developer Conference, all done employing 100% free software: video mixer, encoder, codec and players!
VeeJay over Ethernet
The console interface of FreeJ can be remotely operated using ssh with very good responsiveness even on an internet connection. It can be left running and reconnected later and, last but not least, operated from multiple places at the same time – all usingscreen.
Use all the power and speed of a text console in your video!
This FreeJ console tutorial explores the usage of the console controller (mostly based on the older 0.8 version) and shows you how to load in images and videos, blend them together and put effects on them: it is complete with screenshots and all it needs to start using FreeJ from a console.
This way FreeJ unleashes the power of some GNU/Linux old-school utilities to reach a unique grade of flexibility:
[ascii-box] $ ssh freej-box (authenticate) [freej-box] $ export DISPLAY=localhost:0 [freej-box] $ freej
the freej-box should already be running X on the :0 display, then this will launch a freej running on the remote machine connected to the projector: no more long expensive VGA coaxial cable hassle 😉
Then if you hook up a running freej with a screen -x you can control it from multiple connections at the same time!
Procedural Video Scripting
We can start talking about procedural video as an evolution of the current non/linear paradigm widely spread in video production. In fact, algorithmical approach to video has been widespread since the early ages of the demoscene 😉
A FreeJ scripting API reference is available.
More features in brief…
- 100% FREE GNU GPL Software
- live compositing of multiple webcams, TV signals, movie files, images, TXT files, particle generators and more..
- can be remotely controlled (<b>VJoE</b>)
- can be scripted in procedural object oriented language
- can playback flash vectorial animations
- no frame drop when looping movie clips
- Emacs/Vi style console with hotkeys (S-Lang)
- can accept asynchronous controllers at the same time (Midi, joystick and more coming..)
- very efficient video engine with multithreaded layers
- modular C/C++ code and flexible API
FreeJ is written with efficiency in mind, benefits of a realtime object oriented and multithreaded architecture where layers and controllers all run independently, to take advantage of multiple CPUs and clustered systems.
The language employed in development is C/C++ respecting POSIX compliance and avoiding the computational bloat of some ‘advanced’ C++ functions, which makes it highly portable. The FreeJ Debian package for instance is also distributed in binaries for ARM and MIPS processors.
Its C++ programming API is fairly understandable, here you’ll find a brief introduction to its architecture.
This software is free and open source, you are free to download it, use it, study, modify it and redistribute it, even for commercial purposes, as long as you release your creations the same way, granting your “users” the same rights we grant to you. Share the Freedom! 🙂
For more information see the GNU General Public License.
Below a list of formats you can download this application: ready to be run with some of the interfaces developed, as a library you can use to build your own application and as source code you can study.
FreeJ is easily installed on Ubuntu using its PPA. Use the commands:
sudo apt-add-repository ppa:jaromil/freej sudo apt-get update sudo apt-get install freej
Then proceed having a look at tutorials to get hold of it.
FreeJ is packaged in Arch with a very up to date version. You can see the FreeJ package on AUR and install it from yaourt of just download the PKGBUILD
Here is an overview of packages and quality assurance provided by Debian. Unfortunately the maintainer of FreeJ has horphaned the package, we need help to get it back. This can be done easily because there is already a well done package, while its size is challenging for those who like to learn.
To compile from source on Debian use the following dependencies:
c++-compiler libtool flex bison libsdl-dev libpng-dev libfreetype6-dev libfontconfig-dev dpatch libogg-dev libvorbis-dev libjpeg-dev libslang2-dev libtheora-dev libavcodec-dev libavformat-dev libbluetooth2-dev fftw3-dev libjack-dev libasound-dev liblo0-dev swig python-dev
Apple/OSX 10.5 / 10.6
FreeJ compiles on OSX with developer tools, it has a build project and a native Cocoa GUI written by Xant.
An old build is available as FreeJ.dmg in our download zone.
FreeJ was also included inside an Apple/OSX software called FlowMixer
Nowadays, Xant is working to a new native software for Apple/OSX, still free and open source, called JMX. You’ll hear about it soon!
If you are looking for a winslows version, boot dyne:bolic GNU/Linux!
Latest stable release is 0.10 (30 May 2008).
In our download zone you find all present and past FreeJ releases, source code for extra plugins and more binaries that we occasionally build for various architectures.
A mirror is kindly made available by the Ljudmila medialab.
The bleeding edge version is developed on GitHub/dyne/FreeJ, you can clone the repository, fork and file a pull request.
Please use the version in Git when reporting bugs and getting in contact with us.
Existing graphical interfaces can introduce you quickly to its usage. The FreeJ Manual (PDF) helps you getting started with installation and proceeds introducing you to the power-use of FreeJ via scripting. Of course an “Hello World” example is here
text = new TxtLayer(); text.print("Hello world!"); text.start(); add_layer(text);
You can paste the code above into a file “hello.js” and execute it from commandline or using the “Load script” button from Apple/OSX.
freej -j hello_world.js
Streaming online video
And while streaming, is also possible to save a local copy of the video, all realtime. Here below a script example:
// Audio input is taken via Jack from other applications running // port name buffer size samlerate audio = new AudioJack("alsaplayer", 2048, 44100); // tweak the values below accordingly, see Jack documentation // Create a Video Encoder object // V=video A=audio V quality V bitrate A quality A bitrate encoder = new VideoEncoder(10, 64000, 0, 24000); // Add the audio channel in the video encoded encoder.add_audio(audio); // Configure the encoder to stream over an Icecast server encoder.stream_host("giss.tv"); encoder.stream_port(8000); encoder.stream_title("testing new freej"); encoder.stream_username("source"); encoder.stream_password("2t645"); encoder.stream_mountpoint("freej-test.ogg"); // Register the encoder on the running FreeJ engine register_encoder(encoder); // Start a network stream encoder.start_stream(); // Record the stream into a local file encoder.start_filesave('Video/freej-test.ogg');
You’ll find more complete instructions on our wiki on lab.dyne.org/FreeJStreaming, please feel free to contribute more documentation to that wiki page.
Procedural Video Scripting
Besides the User’s Manual linked above, here below some sources of documentation:
Video manipulation theory
Some interesting links to online publications about video manipulation techniques:
- Introduction to digital image processing
- Image Processing Learning Resources
- Iowa engineering univ. lectures
- YOV 408 technologies
API for C++ programmers
FreeJ is a library that can be linked shared and used by your application. It is fairly easy to be approached by programmers, documented in this simple text file as well in this doxygen code documentation.
The FreeJ source code is mostly written by Denis “Jaromil” Roio, Silvano “Kysucix” Galliani, Christoph “Mr.Goil” Rudorff, Andrea “Xant” Guzzo, Bob “fred_99” Renard, Luca “Shammash” Bigliardi and Filippo “Godog” Giunchedi with contributions by Pablo “Caedes” Martines, Lluis Gomez I Bigorda, Ramiro Cosentino, Tatiana de la O and Andy Nicholson. Check the AUTHORS documentation for complete references.
Parts of libraries are written by Andreas Schiffler (sdlgfx), Jan (theorautils), Dave Griffiths (audio bus), Nemosoft (ccvt), Charles Yates (yuv2rgb), Steve Harris (liblo), Sam Lantinga (sdl*), Jean-Christophe Hoelt (goom), L. Donnie Smith (cwiid), Olivier Debon (flash).
Documentation, testing and user case studies have bee contributed by: Anne-Marie Skriver, Marloes de Valk, Robert de Geus, Piotr Sobolewski, Alejo Duque, Vladimir Flores Garcia and Gabriele Zaverio.
Besides the passionate commitment of its creators, FreeJ development is made possible also thanks to modest funding and infrastructure support by European institutions, organisations and individuals: among them and most importantly are NIMk, Digitale Pioniers and Open Source Alliance.
Please be welcome to join us on the FreeJ’s discussion mailinglist.