First, I have kept my boat painted with Brightsides in the water for as long as a week with no ill effects. My boat has epoxy as a sealer but no glass. Its a lapstrake hull.
If you just have to avoid the paint line: for masking I would do the deck with a coat or two of varnish first to seal it. (esp. if you are painting over wood not epoxy) Then mask off and paint your triim. Finish with additional coats of varnish. Use the 3M Fineline tape.
However, I think it will be easier to do all your varnish coats first then paint your trim. Since you will only need a few coats of paint but many more coats of varnish the process will be easier and the chances of keeping a good edge are better. My boat has painted decks, varnished trim and a color accent. I did the varnish first, then the deck then the trim. The paint edge is not a big deal with the Brightsides since its so thin the line is barely detectable with your fingers. Not at all with your eyes.