August, 2009
BY RICH TOZZOLI
Fast computers, digital video, and powerful DAWs make composing music for the screen easier than ever before.
While traditional music sales may undergoing a time of "transition," one area of the business seems to be more active than ever before: music for picture–be it feature film work, indie documentaries, online video, or commercials. And although the phrase "scoring to picture" may seem technically intimidating, today's blazingfast computers and top-flight software make it easier than ever to get your music in sync with this growing market.
Gearing Up
While you may not need to run out and buy a supercomputer, creating music for picture does require some processing muscle. Most computers available today (or even going back several years) have the ability to not only run your Digital Audio Workstation (DAW), but also a handful of samplers and software instruments– including the orchestral, choir and symphonic libraries often used in film scoring.
Almost all professional-level (and even entry level programs like GarageBand) can import video and sync it with multitrack audio. Since video files tend to be quite large, it's best to use fast external drives (preferably FireWire 800 or SATA) to keep the files off your main drive so it can run the DAW most efficiently. RAM is another important factor: Add as much as you can to your computer (this is a good idea for non-video music production, as well).
That's The Way I Roll
Fundamentally speaking, your job when scoring to picture is to get the video from the client, import it into your system, and "lock" the music down to the frame, so that it's perfectly in sync with the action onscreen. Unless you've also been hired to do the final mix-to-picture, a digital file of your music will be transferred into a video editing system such as Avid Media Composer, Sony Vegas or Apple Final Cut Pro. At that point, it will be re-synced to the picture by the video editors–a process known as layback.
So how do you get started? Well, the first step–aside from the all-important meeting with the filmmakers–is to get the video into your system.
In the past, composers usually received video on tape, and would have to use time code from the tape to sync to their audio recorders–or use an external digitizer and sync box to convert the video and audio on the tape to digital video.
Today, however, producers will likely provide you with a digital version of the video. Most DAWs support digital video import, allowing the video to be viewed (but not edited) within an audio/MIDI project. QuickTime is the most common format because it works with both Mac and Windows platforms, but you may encounter others, such as MPEG, AVI, and even uncompressed digital video (DV). For some of these, you may need to install special software called a CODEC (short for compressor/decompressor) in order to open the file in its original format.
"When dealing with clients, I request a QuickTime with a time code window, sometimes called a 'window burn,'" says awardwinning composer Keith Crane of Sky High Music, whose credits include Sex and the City and spots for Chase, Cover Girl, MasterCard and others, "With longer form projects, I'll either ask for separate cues from editor, or break them up myself for a more efficient work flow. To break up a single movie into smaller files, my assistant or I will import the video into Final Cut or iMovie; from there we'll make new QuickTimes and re-import [each into a project]. Doing that keeps the project more manageable."
Getting Around
Whether you're importing one long video file or separate files for each scene, you need to make sure the music starts at the appropriate point in the video. If the video contains black before the action begins, for example, this may mean that the first important hit does not occur on measure one; it may, in fact be at an awkward point, like the fifth 32nd note of the third beat of measure two. Fortunately, you can use your DAW's video offset feature to adjust the relationship between musical time and video time–for example, setting measure 1 to hit three seconds into the video (Figure 1).
Once it's imported, digital video is extremely convenient to work with. It can be displayed in its own window;as your audio plays, the video will follow. In addition, video can be displayed as a "track" in the Edit/Arrange window. Most DAWs give you the option of showing individual frames– thumbnails of the images in the video; these can make it easier to locate important transitions and scenes. However, if the computer is bogging down, you can save processing power by viewing the digital video in blocks instead of frames (Figure 2). Block mode may not offer the thumbnails, but it lets the processor focus on your audio tracks.
One way to make navigating easier is to set markers at various "hit points." These important locations are often noted by the producer in pre-production meetings, but they may also be something you, as the composer, simply want to accent. One of the challenges in doing music for film is working with your DAW's tempo track so that the timing of the music adjusts to the action on screen. When setting hit points, make sure they lock to time code–as opposed to musical time. This way, any changes to the project's tempo will not affect their location relative to the film.
Today's computers can display video on the same screen as your recording software, but things can get pretty cluttered on bigger projects, which is why Crane opts to separate the digital video fromthe primary production computer. "I use a FireWire Canopus digital video converter that the QuickTime movie goes through, sending out to an external video monitor," he says. "This frees up my computer monitors to be dedicated just for my DAW and related programs and plug-ins."
Audio Considerations
Digital video files provided by producers and video editors often include embedded audio such as dialog and/or temp (temporary) music–tracks you may or may not want to hear while you're working. You can import this audio into your DAW session along with the video, offering you direct control over it; Pro Tools, for example, offers the option of importing the audio from the video file, placing onto a new track, and locating it at the session start time (Figure3).
Sometimes, video producers also provide separate audio mixes with dialog, sound effects (SFX), and temp music, which can all be soloed and muted as needed. Just remember not to include those tracks in your final print when you bounce your music to disk!
Delivery Time
After all the hard work is done importing, syncing and actually scoring the music, you have to get it back to the client. While you can certainly just send an email attachment of a file–or use a service like YouSendIt.com, which allows users to traffic large files–many composers (myself included), choose to work with file transfer protocol (FTP) programs such as Transmit, FileZilla, and Smart FTP. "For finals, I usually send files via FTP," notes Crane. "I may upload a rough mix laid back to the QuickTime itself. Sometimes early on in the composing stage, I might send an MP3 just for client approval of a concept."
Exactly what type of file(s) you deliver for a final product should be discussed in the pre-production meetings. Usually, it's 24-bit files at the highest resolution possible, as Crane noted. "Final deliverables are full bandwidth 48kHz, 24-bit files via FTP or on DVD–depending on where my clients are and how tight of a deadline we're on. Depending on the client's needs, I will either deliver a stereo mix, or break out stems [a mix where individual tracks, or groups of tracks, are printed separately], both with a '2 pop' for sync." (A "2 pop" is a one-second long, 1kHz tone placed two seconds before the start of the audio.)
"TV spots typically get delivered as a simple stereo mix, whereas a longer form documentary or feature will be delivered as both a full 'two-mix' and as stems," Crane explains. "Stems give the director, producers and audio post house the flexibility of having sub mixed tracks to make room for the all important voiceover or dialog. Stems might be broken up as follows: key lead instrument or vocal lines; percussion parts; rhythm section; two or more tracks of orchestra–all separately premixed. Sometimes I will also include the effects, such as reverbs, delays, etc., as separate stems."
Scoring to picture can be extremely challenging, but can also be creatively and financially rewarding. With a little research and some practice, you might be hearing your own work on that new 50-inch HDTV set.
GET IN THE FRAME
While timing is critical in all forms of audio production, it's especially vital when doing sound-for-picture. No matter what video format you're working with, it's extremely important to make sure that everyone on the project is working at the same video frame rate and audio sample rate.
Adding to the problem is the fact that there are several different frame rates, and these may change depending on the type of media the video's derived from. Film uses 24 frames-persecond (fps), but you'll also find 25fps for PAL video (Europe), 29.97fps for NTSC color video, 30fps for black and white TV/audio only, and 23.976fps for HD video. Internet videos? They can use anything!
As with conventional music projects, audio can be at 44.1kHz, 48kHz, 88.2kHz, 96kHz, and 192kHz sample rates, but note that 48kHz is the standard for film and video (as opposed to the 44.1 kHz used on audio CDs).
As far as bit depth, while I prefer to deliver 24-bit audio, many TV/media editors convert everything into 16-bit files for simplicity and storage. Ask your client what they prefer before starting.
So let's say you've established that you'll receive digital video and that you'll need to work at 29.97 fps drop frame time code format, which is used for broadcast TV and for DVDs. Note the term "drop frame"–a small but important issue when working to picture. Drop-frame SMPTE time code skips 18 frames every ten minutes. Time code is measured in Hours: Minutes: Seconds: Frames (which are actually fractions of seconds). But with NTSC video, a frame is not an even fraction of those seconds; it's actually off from real time by 1.8 frames per minute. Thus, drop-frame time code simply tries to adjust for that fact by dropping frames.
01:00:00;00
Note that on the time code display above, there's a semicolon separating the seconds from the frame numbers. This is your clue that you're looking at drop-frame time code. There's also non-drop frame time code, which skips no frames; here, a colon is used between the seconds and frame (01:00:00:00).
Whether or not to use drop or non-drop is usually at the discretion of the video production team: Just make sure you set your DAW or editing system to the same rate the client is using. If you're using drop and the client is not, the audio you produce will eventually go out of sync with the picture.
