Thanks Marcov - I admit I forgot about outputting the generated assembler to a file, and of course the disassembly window, being interpreted from the compiled binary, has all the labels and SEH hints stripped out.
Turns out that you can't insert ".seh_proc" and ".seh_endproc" etc. into a routine, but you can insert ".seh_stackalloc", ".seh_pushreg" and ".seh_savereg" (used if you store a register at a particular point in the stack frame rather than using "push"). It seems this was the answer I was looking for. The only drawback is that this is for AT&T syntax, and I don't think there's an equivalent yet available for Intel syntax.
ADDENDUM: If you use these, you also need to put ".seh_endprologue" at the very start of the procedure if you specify the "nostackframe" directive.