I faced this exact risk a year ago. I didn't want to stay in a hotel in NYC (they are so expensive and tiny) so I read up about private apartment rentals.
Yes it's illegal.
Yes you may be thrown out on the street during your stay.
That being said, there are lots of private rentals in NYC - I picked one of the many places on airbnb and had a terrific experience. I knew I could be kicked out at any time but took the risk and it was worth it. The single most important thing to remember if you dont want to be kicked out is to ensure no-one else in that building realises you are holiday renting - that is how people get caught, when the doorman/other residents realise an apartment in their building is being used like a hotel - they get annoyed from a security perspective, dob you in and you get kicked out. This means minimal communication with the doormen or anyone else in the building (I was putting on a NYC accent within a few days of staying there to blend in). Know the name of the person who lets the apartment and if anyone asks you are a relative/close friend visiting that person for a short time, and that you are NOT sub-letting. It's not illegal to let someone stay with you for free, it is to sub-let to a stranger. No-one asked but I had the story ready if they did.
It's risky. For some people it works and for some people it doesn't. If you don't like risks stay in a hotel, pay more for less room and relax. If you want a bit of an adventure and want to live like a local hire a private apartment and take the risk it may not work out. It worked for me (was one of the best parts of the trip) and I wouldn't hesitate to do it again, but that's the sort of person I am.