I found it easy to remove the clip, but yes... You should unload it first. Doesn't take a lot of force.
(Alone) I found the easiest way was to place the column against the ground and grab the column housing (outside) with one hand and press down against the ground, thereby compressing the spring a little. Been a while since I've been in there, but I probably braced the whole thing against a hip or something? Anyway, at that point you can remove the clip as soon as it's unloaded. Of course, if you have a helper with a second pair of hands, it's even easier.
Some pics because we all like pics: