Tag: vj

Entries for tag "vj", ordered from most recent. Entry count: 5.

Pages: 1

# I Implemented Video and GIF Playback with FFmpeg

Thu
27
Apr 2017

I started my music visualization project with a possibility to load a collection of static images, accompanied with metadata that describe their parameters like dominant colors, whether it uses alpha transparency or whether it is tileable (description of these parameters could be a topic of another blog post). I then render them as textures, moving, rotating and changing colors randomly, blended together, with a possibility of feedback from previous frame. This mathod may sometimes generate quite interesting images, but it has its own limitations and becomes boring after some time.

Generating some interesting graphics procedurally from scratch is my ultimate goal. But while writing shaders is fun and can give amazing results (as we can see on ShaderToy), it's also the hard way. So to be able to show something interesting, for now I've implemented playback of videos and animated GIFs, using FFmpeg library.

FFmpeg is a free tool that contains its own codecs for various video and audio formats (so it doesn't use the codecs installed in Windows). It is known as a command-line program that can convert about any video format, but it's also a software library that offers this encoding/decoding features to developers. I learned to use this library because I needed to implement video playback for one of my shows.

Later I discovered that it can play animated GIFs as well. This is a great feature, because having hundreds of such GIFs downloaded and being able to switch between them in an instant can make quite interesting visuals. There are many abstract, geometric, psychedelic animations shared all around the Internet, like on Op Art or Fractalgasm Facebook pages. At the same time, possibility to play all popular video formats is much more comfortable than what Resolume offers, which requires converting all the footage to its own codec, called DXDI (by the way, FFmpeg is able to play this as well).

I won't show any source code this time, but if you are a developer and consider implementing support for video playback or encoding, I recommend FFmpeg library. Other option is libVLC - a library behind popular VLC media player, which also has its own set of codecs. I also used it some time ago. Playing anmimated GIF-s is also possible through Windows Imaging Component (WIC), which is part of standard Windows API.

Comments | #vj #video #graphics Share

# Sztukato 2016 - Festival of Arts and Fashion

Thu
31
Mar 2016

18-20 March 2016 in Protokultura club in Gdańsk, Poland, an interesting event took place: Sztukato - festival of arts and fashion (Website, Facebook Event). It involved arts gallery, fair of handmade clothes and accessories, fashion shows, concerts and many other activities. I was doing visualizations during the whole event. It was new and interesting experience for me, as I learned a lot during the event, as well as while preparing for it. I especially gained lots of experience in video editing, as I prepared some prerendered video footage. Depending on the circumstances sometimes I played these videos in a loop, sometimes just showed logos of organizers and sponsors and sometimes launched the abstract/psychedelic visuals generated procedurally by my program.

Here is full gallery of my photos from the festival: SZTUKATO 2016 Festiwal Sztuki i Mody @ Facebook.

I can see many VJ-s use Resolume, but for simple displaying images or videos I used Screen Monkey. It's a free program that I came across when browsing VJ Forums. It has some problems (GUI has some minor bugs and it even stops playing videos sometimes), but it also has many useful features (layers, fade in/out, linking clips in a sequence, Schedule and many more).

The biggest problem I had with Screen Monkey is that it didn't want to play any videos after installation. (My environment was: Windows 7 x64 with latest updates, K-Lite Codec Pack Full in latest version, Screen Monkey version 3.7, video files format: MP4 container + MPEG4 Video (H264) video stream) Solution to this turned out to be:

1. Add any video clip to Screen Monkey Dashboard.
2. Right-click on this clip, select: Video Options.
3. Uncheck: Prefer ffdshow Video Decoder.

After going back to my work, I had a thought that there is one big difference between creative work and doing software engineering. When creating something, whether it's an art, writing a book or even coding a small program, you can always come up with SOMETHING even if you lack knowledge, experience or time and the deadline is close. It may be better or worse, client may like it or not, but at least you have SOMETHING and the rest is just a matter of negotiation. When working in software, it's more binary - all-or-nothing. You either meet the specification or not, pass all unit tests or not, you fixed the bug or not. Sure you can also write better or worse code, your solution can be more robust, efficient or better architected, but this has its own problem: Writing bad code increases technical debt, which makes it harder to work with the code in the future while being quite invisible to the client and your manager. On the other hand, when assigned some creative task, you probably launch your editor and start from a blank document every time.

Comments | #events #vj Share

# Music Visualizations - Plans for the Future

Mon
14
Sep 2015

Last weekend I was showing my music visualizations on two parties. First one was in a club, projected on flat screen, while the second one was an open-air party (my 7th and last one in this summer season :) with the image projected on trees. For this one I had to prepare something different - simple, contrast shapes and single color only, so it could be clearly visible. Here is a small video:

It doesn't look like this because I wanted it exactly this way or because that was my "artistic vision", but just because showing some rotating images downloaded from the Internet and blending transformed feedback from previous frame was the easiest way to start with something interestingly looking.

Now I have tons of ideas to improve this program as soon as I find some free time. Next to some small technical tasks like refactoring code or simply adding new graphical effects, I plan following big TODO-s (with no particular order decided yet):

• Video playback. Being able to show video streams instead of still images would be great, but is also hard to implement. I plan to use FFmpeg library for this purpose. There are other options as well. I'm afraid of performace of uploading each decoded frame from CPU to GPU, so maybe using Microsoft Media Foundation/DXVA (it seems to work with Direct3D 11) or DirectShow would be better, but on the other hand, codecs for various file formats could be a problem, while FFmpeg has its own codecs.
• A second window with GUI to be displayed on laptop's screen, because right now I have only standard Windows console with output done using printf() and input taken from console commands (which is not a convenient way to control the program). Next to some buttons and other controls, an important feature would be to display in this window a small copy of the final image presented on the projector (so the operator doesn't need to turn around to see it) and some technical parameters shown in real-time (like current FPS, audio UV meter).
• Music analysis. Right now I have some basic audio input implemented using Windows Multimedia/WaveIn. I want to switch to Core Audio APIs/WASAPI. Currently I only measure peak amplitude and visualize it on this shape in the center, but I could extract much more information from the audio input (perform FFT, detect beat), control more parameters with it (like alpha transparency of some image layers) and try to automatically synchronize visuals to the beat.
• Some framework for parameters of different types (bools, ints, floats, vectors, strings, sub-containers etc.), where each one can be serialized to/from a file and exposed to GUI for editing. I guess all progammers love to implement this :) I also did it multiple times already, but this time it will be the ultimate, universal, all-encompassing solution ;) I want parameters to have constant value, to be animated in time according to some curve or to be evaluated from a function depending on some variables, like the data from music input. This way I could use the framework to describe elements of a scene (parameters like position, size, orientation, colors etc.). I already have such system implemented to some degree, but it definitely could be improved.
• Support for some input device more convenient than laptop touchpad and keyboard, like a MIDI controller (or a smartphone or tablet simulating one) or... a Wiimote :)
• Asynchronous loading of resources, like textures, in the background, on separate thread. Because I know one day all my resources won't fit into GPU memory and I will have to load them on the fly to prepare next scene while showing current one.
• Video capture from a camera (or better camera with depth, like Kinect), plus maybe some processing with computer vision algorithms (like in OpenCV library).

Comments | #events #music #psytrance #vj Share

# Psychill Evening Vol 2 with my visuals

Fri
06
Feb 2015

On Friday, 13 February 2015 I'd like to invite you for second edition of Psychill Evening party in Paszcza Lwa club in Gdańsk, Poland. Once again the music will be "Psychill, Psybient, Downtempo, Psydub and Ambient", so basically more calm genres of electronic music. Once again I will be doing visuals on this party. See also my blog entry and video below from first edition.

I'm now working on my software to prepare some new stuff. Some random screenshots:

Of course this is a blog about programming, not about art or parties, so in the next posts I will explain some of the technical details behind this program.

Comments | #events #music #psytrance #vj Share

# PsyChill Evening - my first music visualizations

Sat
22
Nov 2014

Yesterday I had an opportunity - for the first time - to show my visualizations as a VJ on a music party. The party was called PsyChill Evening, took place in Paszcza Lwa, Gdańsk, Poland and its musical style was described as "Psychill, Psybient, Downtempo, Psydub and Ambient".

Just like DJ is someone who plays music, a VJ is an auxilliary role of someone who makes visualizations in real-time, displayed using a beamer. There is some software dedicated for this, like Resolume or ArKaos, but what can be more fun for a graphics software enginner than coding my own software? So I made one. Honestly, I planned to write program like this for years, while now I had to do it in just one week, right before the party. It's written in C++ using DirectX 11. Here is how the result looks like:

As you can see, the rendering here is not very sophisticated, CPU- or GPU-intensive. There are just some 2D textures transformed and blended together (plus feedback from previous frame, plus symmetry) - all fading in and out smoothly, as it was for chillout music. But that's not the point here. There is some interesting code under the hood, like the way a "scene" is described in memory with all these changes that happen over time. Time is expressed inside the program not in seconds, but in beats, so after setting right BPM (Beats Per Minute), it synchronizes nicely with the music. The movement of these textures, as well as color transformations are all procedurally generated and random, so it's somewhat different every time. Of course, there is much more to be done here. But now all I need is some sleep :)

I love electronic dance music, so I'm very happy I could connect it wih my profession - graphics programming :D

Comments | #music #psytrance #vj #events Share

Pages: 1