The difference would be being able to pick the clients and projects that are truly interesting/exciting and being able to work without all the stress.
Exactly. If you currently enjoy what you do then it would only be better because you know that at any time you can end it if it gets too stressful.
Yeah, but if I did work, it'd be something I'd want to do.
That would be the key. They money doesn't matter as long as you are happy with what you do. I've lived that way since my first 'official' job at 14. That explains why I had 23 jobs by the time I was 18 but I was never a day without work (I never quit before I had another one lined up) and I was never fired. Once I stopped being challenged by the job I moved on.