Not Sure Whether to Laugh or Cry
I shouldn't laugh, but this is funny. RCIA itself, given its sad state at many parishes, might serve as both initiation and goodbye. (I speak from personal knowledge; a co-worker's relapse back into the Church was going well until the utter insensitivity of the leader of RCIA helped provoke a re-relapse.)
Anyway, as for a ritualistic song of depature, I've always been partial to "Goodbye Ladies", the song we sing at family gatherings when someone is eliminated from a card game (substituting the person's name for 'ladies' of course).