Consider this: You commision a composer to write a personal song to celebrate you at your anniverseray. You use the song, but decline to pay because the composer can keep a copy of the song, and therefore you are not really taking anything from him. But clearly you have put him in a worse position by using his work without pay - after all he could have done something else with his time and talent.
Interesting hypothetical, but I think there is a difference between specifically asking for a service (and possibly creating a contract) and not paying vs sharing a copy of a work already produced. Otherwise, there are loads of services you can argue against paying for in that way.
Perhaps it was a bad example, but my point was that the argument that "piracy is not taking anyting away" assumes that IP is created in a vaccuum without consideration of the potential audience. This is rarely the case. Production of movies, music etc. is a risky investment with the expectation that if people enjoy the product they will pay. If nobody enjoys it, fair enough, bad investment. But if people do enjoy it, but still don't pay, clearly this is harming the creators.
