I'd just paint over it...

Okay, I had two different brands of PVC, so I experimented. On one of them, mineral spirits started to remove the lettering didn't get it all. On the other, mineral spirits did nothing. Denatured alcohol does nothing. Acetone crazes the surface. 1600 grit wet sandpaper did it.

I just went thru this, I used bleach with ok results, then on another piece I tried denatured alcohol , finally resorted to the best solution, plastic spray paint.

Acetone dissolves plastics, and it's extremely poisonous vapors.
don't use acetone without ventilation, you shouldn't even use acetone due to it's extreme toxicity when it
evaps into the atmosphere, bad bad chemmie.