I'm using iMac Os X 10.5.8. I'm trying to record some clips of films that I made to create a demo reel. I'm trying to record clips, export the .mov to the desktop, bring into FCP, edit, export to Vimeo.
When I open Viola everything is fully functional. I can open and click everything just fine. I click the options arrow above the "record" function with the green camera. I click on "selection". It then gives me the option to "click and drag" the selected area I'd like to record.
Regarding your query, once done with recording. You can stop the recording (either save or cancel) in the menu bar, please refer the screen shot below.
Also you can stop video recording using the hotkey i.e "cmd+shift+esc".