27 1 / 2012

Seriously.js: A Realtime, Node-Based Video Compositor for the Web

We covered a demo earlier this week showing real time video processing with WebGL, but Seriously.js taking this concept even further: seriously!

Seriously.js is a real-time, node-based video compositor for the web. Inspired by professional software such as After Effects and Nuke, Seriously.js renders high-quality video effects, but allows them to be dynamic and interactive.

It features an optimized rendering path that is built with WebGL making it entirely hardware accelerated at 60 frames per second.  You can use videos, images, canvases and pixel arrays as input sources to the compositor and then apply various video effects to the resulting image, and apply 2d transforms to any layer (3d is coming).  The available effects are numerous ranging from chroma key and night vision to edge detection and other common effects.  Additionally, and perhaps the best feature of the library is that you can add your own effects as plugins and use them just as you would the built in effects… In fact, the built in effects are actually just plugins themselves!

The demo combines a number of different effects and sources.  The video is chroma-keyed so that you can actually change the background as it’s playing in real time, and then various effects are applied on top of everything changing the feel of the music video dramatically.  There is a underwater scene, a night vision scene, and a creepy black one.  I think this idea of user modifiable music videos is a very neat one indeed!

So want to get started with Seriously.js yourself?  Luckily for you, there is a tutorial to help you out and even some documentation!  So how easy is it to apply effects to an HTML5 video?  This easy:

var seriously = new Seriously(),
    video = seriously.source('#myvideo'), // get video element by CSS selector
    target = seriously.target('#canvas'), // output canvas
    vignette = seriously.effect('vignette');

// connect all our nodes in the right order
vignette.source = video;
target.source = vignette;
seriously.go();

Obviously you can connect multiple nodes together in a chain-like fashion, adjust effect parameters and more, so check out the tutorial for all of that!

Who knew that JavaScript was going to be doing realtime video compositing and effects even a few years ago?  Seriously.js is seriously impressive, it is open source on Github, and you should check it out… seriously!

Tags:

Permalink

04 8 / 2010

GLGE: Taking the pain out of WebGL

GLGE is a library for WebGL that aims to “mask the involved nature of WebGL from the web developer, who can then spend his/her time creating richer content for the web.”  It has a pretty impressive list of features:

  1. Keyframe animation
  2. Perpixel lighting directional lights, spot lights and point lights
  3. Normal mapping 
  4. Animated materials
  5. Skeletal animation
  6. Collada format support
  7. Parallax Mapping
  8. Text rendering
  9. Fog
  10. Depth Shadows
  11. Shader based picking
  12. Environment mapping
  13. Reflections/Refractions
  14. Collada Animations
  15. 2D Filters
  16. Video and Canvas Textures
  17. …And many more!

Dennis Ippel has written a getting started guide for using GLGE, which shows just how easy it is to create your very own 3d scene using data exported from Blender and a bit of XML.

<?xml version="1.0" ?>
<glge>
	<mesh id="box">
		<positions>1.000000,0.999999,1.000000,1.000000,1.000000,-1.000000,-1.000000,1.000000,-1.000000,1.000000,0.999999,1.000000,-1.000000,1.000000,-1.000000,-1.000000,1.000000,1.000000,-1.000000,-1.000000,-1.000000,-1.000000,-1.000000,1.000000,-1.000000,1.000000,1.000000,-1.000000,-1.000000,-1.000000,-1.000000,1.000000,1.000000,-1.000000,1.000000,-1.000000,1.000000,-1.000000,-1.000000,0.999999,-1.000001,1.000000,-1.000000,-1.000000,-1.000000,0.999999,-1.000001,1.000000,-1.000000,-1.000000,1.000000,-1.000000,-1.000000,-1.000000,1.000000,1.000000,-1.000000,1.000000,0.999999,1.000000,1.000000,-1.000000,-1.000000,1.000000,0.999999,1.000000,0.999999,-1.000001,1.000000,1.000000,-1.000000,-1.000000,1.000000,0.999999,1.000000,-1.000000,1.000000,1.000000,0.999999,-1.000001,1.000000,-1.000000,1.000000,1.000000,-1.000000,-1.000000,1.000000,0.999999,-1.000001,1.000000,1.000000,1.000000,-1.000000,1.000000,-1.000000,-1.000000,-1.000000,-1.000000,-1.000000,1.000000,1.000000,-1.000000,-1.000000,-1.000000,-1.000000,-1.000000,1.000000,-1.000000</positions>
		<normals>0.000000,1.000000,0.000000,0.000000,1.000000,0.000000,0.000000,1.000000,0.000000,0.000000,1.000000,0.000000,0.000000,1.000000,0.000000,0.000000,1.000000,0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-0.000000,-1.000000,-0.000000,-0.000000,-1.000000,-0.000000,-0.000000,-1.000000,-0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,1.000000,0.000000,-0.000000,1.000000,0.000000,-0.000000,1.000000,0.000000,-0.000000,1.000000,-0.000001,0.000000,1.000000,-0.000001,0.000000,1.000000,-0.000001,0.000000,-0.000000,-0.000000,1.000000,-0.000000,-0.000000,1.000000,-0.000000,-0.000000,1.000000,0.000000,-0.000000,1.000000,0.000000,-0.000000,1.000000,0.000000,-0.000000,1.000000,0.000000,0.000000,-1.000000,0.000000,0.000000,-1.000000,0.000000,0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000,0.000000,-0.000000,-1.000000</normals>
		<uv1>0.333333,0.498471,0.001020,0.500000,0.000000,0.001529,0.333333,0.498471,0.000000,0.001529,0.332314,0.000000,0.000000,0.998471,0.001020,0.500000,0.333333,0.501529,0.000000,0.998471,0.333333,0.501529,0.332314,1.000000,0.665647,0.500000,0.666667,0.998471,0.333333,0.501529,0.666667,0.998471,0.334353,1.000000,0.333333,0.501529,0.998981,0.000000,1.000000,0.498471,0.666667,0.001529,1.000000,0.498471,0.667686,0.500000,0.666667,0.001529,1.000000,0.501529,0.998981,1.000000,0.667686,0.500000,0.998981,1.000000,0.666667,0.998471,0.667686,0.500000,0.334353,0.500000,0.333333,0.001529,0.665647,0.000000,0.334353,0.500000,0.665647,0.000000,0.666667,0.498471</uv1>
		<faces>0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35</faces>
	</mesh>

	<camera id="maincamera" loc_z="20" />

	<material id="boxmaterial" color="#900" />

	<scene id="mainscene" camera="#maincamera" ambient_color="#fff">
		<light id="mainlight" loc_y="5" type="L_POINT" />
		<object id="box" mesh="#box" rot_x="-.8" rot_y=".5" material="#boxmaterial" />
	</scene>
</glge>

Then, load and render the scene using JavaScript.

var doc = new GLGE.Document();

doc.onLoad = function() {
	var renderer = new GLGE.Renderer(document.getElementById("canvas"));
	var scene = doc.getElement("mainscene");
	renderer.setScene(scene);

	//render each millisecond
	setInterval(renderer.render, 1);
}

// load the xml containing the scene data
doc.load("scene1.xml");

After adding some simple textures and animations to the XML scene definition, you get something looking like this:

As you can see, GLGE makes it extremely easy to get started with writing 3d hardware accelerated 3d applications in JavaScript.  Awesome work!

You can check out lots more demos on the GLGE website, the code on Github, and Dennis Ippel’s fantastic GLGE tutorial here.

Tags:

Permalink

19 6 / 2010

Visualizing and Manipulating Audio in the Browser

The Firefox team has been busy working on the HTML5 Audio Data API, which lets JavaScript code interact with the Audio tag in a bidirectional manner.  Developers have been busy putting together some cool demos of what the API enables, including visualizations using the Canvas element, as well as playing interactive audio generated by JavaScript itself.

Jacob Seidelin has written some Winamp style visualizations for live audio playing in the browser.  If you are not running the experimental version of Firefox that this API is a part of, the audio playback will fall back to Soundmanager2 (Flash audio).  This shows what is now possible in the browser in real time.  I hope that other browsers implement the API soon!

Another demo, this time using WebGL for hardware-accelerated 3d graphics, was written by Charles Cliffe and does beat detection.  It uses the CubicVR 3D engine built on WebGL as well as Processing.js for the rendering.  Very cool!

At the beginning of this post, I mentioned that the API worked in the opposite direction as well: audio output from JavaScript.  One of the demos showcasing this technology, was written by Alistair MacDonald, and is an instrument inspired by Brian Eno’s Bloom iPhone application.  Check out the demo video, below.

Finally, check out this text-to-speech engine written in JavaScript using the Mozilla Audio API.  It is a real showcase of what the browser is now able to do.  Awesome work, everyone.  For more demos, check out this post on the Bocoup blog by Alistair MacDonald.  JavaScript FTW!

Tags:

Permalink