How do I... Create an image mask when using Sony Vegas?

Sony Vegas is an image editing and manipulation application geared toward the average amateur editor, but it is still a very powerful and capable tool. Creating what is commonly known as a mask in Sony Vegas requires the use of an applet called Cookie Cutter. Steven Warren shows you how to create a mask using this tool inside of Sony Vegas.

I received a call recently from someone wanting to know how to create an image mask using Sony Vegas. Since I am a huge proponent of Sony Vegas, I embraced this opportunity.

The question was actually a very good one because the creators of Sony Vegas decided not to call it a masking tool like other image editing applications. Rather, it chose to call its tool Cookie Cutter.

Sony Vegas is a consumer product made for the average amateur editor, but when it comes to overall functionality, Vegas rivals its more extensive and expensive competitors. Keeping that little tidbit in mind let us proceed to the tutorial.

This blog post is also available in PDF form as a TechRepublic download.

Creating a mask

Let's start by opening a new document by going up to File on the menu and then New, or the [Ctrl] + N keyboard shortcut (Figure A).

Figure A

New file
Once you open your new file, the next step is to browse for content you will be using. I will be using a simple JPEG for this example (Figure B).

Figure B

An example JPEG

This technique will work for static images as well as your stock video footage. I will talk a little more about other features of the masking tools for video footage later.

After choosing your media, drag out onto the timeline (Figure C).

Figure C

Drag to the timeline

Proceed to the bottom tabs and select Video FX. The FX you are looking for in the list is Cookie Cutter.

Now, let me take a minute to explain what a mask really is. As amazing as it may sound, masking is a technique that has been around since before computers. A mask, put simply, cuts out the junk that you do not want the viewer to see.

For example, if you have only a small green screen and you want a wider shot of the subject, you can mask out all of the surrounding background that is not key-able. Keep in mind that the subject must stay in front of the green screen.

Another example would be if you had a subject like a person and another object that's an element like a digital effect that you want to pass behind the subject. Using Adobe After Effects you can insert a mask and custom shape the mask around your subject. (Unfortunately, you can't do this with Sony Vegas.) By doing this you give the impression that the digital effect is passing behind your subject.

Okay, that is enough of that. After choosing the mask or cookie cutter shape you want, drag it onto your media. A dialog box will pop up with the default effect settings (Figure D).

Figure D

Default settings
Once that has opened you can make your adjustments. If you accidentally close your properties dialog window you can bring it back up by clicking on the green puzzle piece located on your media in the timeline, as shown in Figure E.

Figure E

Green puzzle piece

I have created a new layer to go beneath my original layer. It is easy to see the power of masking tools when used in your compositions. As I mentioned earlier, more advanced programs like After Effects have more customization options with masks, but Sony Vegas does a decent job of introducing masks.

Some of the default settings include a border around the mask indicating the size and shape of your mask. This may or may not be helpful to what you are trying to accomplish in your composition, but this is one of the settings you can turn off.

When you first apply the effect to your composition and you can't see your subject, your mask may be positioned incorrectly. At the bottom-left corner of your properties dialog box you will see a white palette that represents the composition. On that palette there will be a square and when you drag your cursor over the palette a grab hand will appear, as shown in Figure F.

Figure F

A grab hand over the bottom-left corner

The square indicates the center position of your mask shape. Grabbing the square with the hand and dragging it around the palette will help reveal your missing subject.

There is one more tip I have yet to cover. The Cookie Cutter tool is key frame-able. What I mean by this is that you can do simple animation of the mask's position using the key frame timeline at the bottom of your properties dialog window. Double-click with the mouse at the time you wish to start the animation. A diamond will appear and that is your key frame (Figure G).

Figure G

Key frame

Once you have created your starting key frame, click later in the timeline where you want to end the animation. Go to your position palette and move it to the desired position. That is it -- a pretty simple concept and very effective. Now that you have learned the secrets of masking with the Cookie Cutter function in Sony Vegas, try using it in your next comp and don't be afraid to experiment.