FreeJ is a vision mixer: an instrument for realtime video manipulation used in the fields of dance teather, veejaying, medical visualisation and TV. Its development started in 2001 and continued actively for many years.
What we need to do
We need to interact with multiple layers of video, filtered by effect chains and then mixed together.
We want to write scripts to control video mixes with keyboards, midi signals, OSC messages, wiimotes, video mouse and joysticks; manipulating images, movies, live cameras, particle generators, text scrollers, flash animations and more.
We show the resulting video mix on multiple and remote screens, encode it into a movie and stream it live to the internet.
How FreeJ does it
FreeJ is a commandline application on GNU/Linux, a graphical application on Apple/OSX, a C++ library offering an API for a multimedia framework that relies on different native functions on the operating systems it is ported and, at last, bindings to languages like Python and Ruby (using Swig, more can be implemented as needed.
The code is fairly documented and usable in C++, with full bindings to python, while parsing scripts is done using Spidermonkey, the Mozilla interpreter.
FreeJ has started being developed on a dual-core CPU already in 2001 and has grown with emphasis on multi-threading to run efficiently on modern multi-core computers.
With FreeJ we can overlay, mask, transform and filter multiple layers on the screen. There is no limit to the number of layers that can be mixed. Each layer can be video taken from different sources: movie files, webcams, tv cards, images, rendered text,flash animations and more: even particle generators are there, and effects from the frei0r plugin collection.
Stream online video
long live pirate TV! scream your video to the masses!
FreeJ can produce an Ogg/Theora/Vorbis stream for broadcast on Icecast servers, mixing all the video and grabbing the audio from soundcard. The resulting video can be played on any computer connected via internet (list of compatible players).
Instructions on how to stream with FreeJ.
Here are the recordings of a streaming veejay session by Kysucix, screened live at the Linux Audio Developer Conference, all done employing 100% free software: mixer, encoder, codec and players! download the free footage for a demo here ftp.dyne.org/performance/freej-lad2005
VeeJay over Ethernet
The console interface of FreeJ can be remotely operated using ssh with very good responsiveness even on an internet connection. It can be left running and reconnected later and, last but not least, operated from multiple places at the same time – all usingscreen.
Use all the power and speed of a text console in your video!
This FreeJ console tutorial explores the usage of the console controller (mostly based on the older 0.8 version) and shows you how to load in images and videos, blend them together and put effects on them: it is complete with screenshots and all it needs to start using FreeJ from a console.
This way FreeJ unleashes the power of some GNU/Linux old-school utilities to reach a unique grade of flexibility:
[ascii-box] $ ssh freej-box (authenticate) [freej-box] $ export DISPLAY=localhost:0 [freej-box] $ freej
the freej-box should already be running X on the :0 display, then this will launch a freej running on the remote machine connected to the projector: no more long expensive VGA coaxial cable hassle
Then if you hook up a running freej with a screen -x you can control it from multiple connections at the same time!
Procedural Video Scripting
We can start talking about procedural video as an evolution of the current non/linear paradigm widely spread in video production. In fact, algorithmical approach to video has been widespread since the early ages of the demoscene
A FreeJ scripting API reference is available.
More features in brief…
- 100% FREE GNU GPL Software
- live compositing of multiple webcams, TV signals, movie files, images, TXT files, particle generators and more..
- can be remotely controlled (<b>VJoE</b>)
- can be scripted in procedural object oriented language
- can playback flash vectorial animations
- no frame drop when looping movie clips
- Emacs/Vi style console with hotkeys (S-Lang)
- can accept asynchronous controllers at the same time (Midi, joystick and more coming..)
- very efficient video engine with multithreaded layers
- modular C/C++ code and flexible API
FreeJ is written with efficiency in mind, benefits of a realtime object oriented and multithreaded architecture where layers and controllers all run independently, to take advantage of multiple CPUs and clustered systems.
The language employed in development is C/C++ respecting POSIX compliance and avoiding the computational bloat of some ‘advanced’ C++ functions, which makes it highly portable. The FreeJ Debian package for instance is also distributed in binaries for ARM and MIPS processors.
Its C++ programming API is fairly understandable, here you’ll find a brief introduction to it.
This software is free and open source, you are free to download it, use it, study, modify it and redistribute it, even for commercial purposes, as long as you release your creations the same way, granting your “users” the same rights we grant to you. Share the Freedom!
For more information see the GNU General Public License.
Below a list of formats you can download this application: ready to be run with some of the interfaces developed, as a library you can use to build your own application and as source code you can study.
On ftp.dyne.org/freej you find all present and past FreeJ releases, source code for extra plugins and more binaries that we occasionally build for various architectures.
A mirror is kindly made available by the Ljudmila medialab.
The bleeding edge version is developed on our code repository using git, you can clone the repository free and anonymously
git clone git://code.dyne.org/freej.git
please use this version when reporting bugs and getting in contact with us.
packages in Debian are updated frequently, to compile from source use the following dependencies:
sudo apt-get install c++-compiler libtool flex bison libsdl-dev \ libpng-dev libfreetype6-dev libfontconfig-dev dpatch libogg-dev \ libvorbis-dev libjpeg-dev libslang2-dev libtheora-dev \ libavcodec-dev libavformat-dev libbluetooth2-dev fftw3-dev \ libjack-dev libasound-dev liblo0-dev swig python-dev
Apple Darwin/OSX 10.5 / 10.6
Download either the i386/PPC or the x8664 (snow leopard only) FreeJ with support for quicktime, freeframe and frei0r effect plugins here: this version includes an experimental graphical interface (Carbon/Cocoa) that shows functionalities.
We are looking for designers to help make it look better! please contact one if you like to be the one
Here is an overview of packages hosted by Ubuntu. Using Synaptic, simply install freej (or apt-get install freej of course).
If you like, you can also download our packages:
or add “deb http://ppa.launchpad.net/jaromil/ubuntu hardy main” to your /etc/apt/sources
FreeJ is packaged in Arch since long time and has been just updated recently, you can see the FreeJ package on AUR and install it from yaourt of just download the PKGBUILD
If you are looking for a winslows version, boot dyne:bolic GNU/Linux!
Existing graphical interfaces can introduce you quickly to its usage. The FreeJ Manual (PDF) helps you getting started with installation and proceeds introducing you to the power-use of FreeJ via scripting. Of course an “Hello World” example is here
text = new TxtLayer(); text.print("Hello world!"); text.start(); add_layer(text);
You can paste the code above into a file “hello.js” and execute it from commandline or using the “Load script” button from Apple/OSX.
freej -j hello_world.js
Streaming online video
And while streaming, is also possible to save a local copy of the video, all realtime. Here below a script example:
// Audio input is taken via Jack from other applications running // port name buffer size samlerate audio = new AudioJack("alsaplayer", 2048, 44100); // tweak the values below accordingly, see Jack documentation // Create a Video Encoder object // V=video A=audio V quality V bitrate A quality A bitrate encoder = new VideoEncoder(10, 64000, 0, 24000); // Add the audio channel in the video encoded encoder.add_audio(audio); // Configure the encoder to stream over an Icecast server encoder.stream_host("giss.tv"); encoder.stream_port(8000); encoder.stream_title("testing new freej"); encoder.stream_username("source"); encoder.stream_password("2t645"); encoder.stream_mountpoint("freej-test.ogg"); // Register the encoder on the running FreeJ engine register_encoder(encoder); // Start a network stream encoder.start_stream(); // Record the stream into a local file encoder.start_filesave('Video/freej-test.ogg');
You’ll find more complete instructions on our wiki on lab.dyne.org/FreeJStreaming, please feel free to contribute more documentation to that wiki page.
Procedural Video Scripting
Besides the User’s Manual linked above, here below some sources of documentation:
Video manipulation theory
Some interesting links to online publications about video manipulation techniques:
- Introduction to digital image processing
- Image Processing Learning Resources
- Iowa engineering univ. lectures
- YOV 408 technologies
API for C++ programmers
FreeJ is a library that can be linked shared and used by your application. It is fairly easy to be approached by programmers, documented in this simple text file as well in this doxygen code documentation.
The FreeJ source code is mostly written by Denis “Jaromil” Roio, Silvano “Kysucix” Galliani, Christoph “Mr.Goil” Rudorff, Andrea “Xant” Guzzo, Luca “Shammash” Bigliardi and Filippo “Godog” Giunchedi with contributions by Pablo “Caedes” Martines, Lluis Gomez I Bigorda, Ramiro Cosentino, Tatiana de la O and Andy Nicholson. Check the AUTHORS documentation for complete references.
Parts of libraries are written by Andreas Schiffler (sdlgfx), Jan (theorautils), Dave Griffiths (audio bus), Nemosoft (ccvt), Charles Yates (yuv2rgb), Steve Harris (liblo), Sam Lantinga (sdl*), Jean-Christophe Hoelt (goom), L. Donnie Smith (cwiid), Olivier Debon (flash).
Documentation, testing and user case studies have bee contributed by: Anne-Marie Skriver, Marloes de Valk, Robert de Geus, Piotr Sobolewski, Alejo Duque, Vladimir Flores Garcia and Gabriele Zaverio.
Besides the passionate commitment of its creators, FreeJ development is made possible also thanks to modest funding and infrastructure support by European institutions, organisations and individuals: among them and most importantly are NIMk, Digitale Pioniers and Open Source Alliance.
If you are new to mailinglists, please take your time to read those Netiquette Guidelines (RFC1855) and in particular the chapter 3.0 about One-to-Many Communication.
The mailinglists we use for discussion are completely free, they are published on the web and can be searched in publicly available archives. Posting to them is limited to subscribers and subscription is free.
Please do not send bugreports on any mailinglist, use instead our bugtracker which helps tracking down the reports and keeping them in order.
Your email details will be never disclosed: neither in the public mail archives, neither will be sold to anyone. This service is provided “as is”, in the hope it will be useful but without any other warranty.
Anyone posting is the sole responsible for any information shared.