Depends on jurisdiction, but I think in most countries it's a crime to not adhere to aviation safety regulations - including ignoring what the staff tells you to do in an emergency.
Smoking in the lavatories, or standing up and refusing to sit down when you don't want to travel with someone - that results in criminal liability.
There was a report of people being taken off a flight a few days ago for refusing to listen to the safety announcements. They were sitting in an exit row.
There are also several cases over recent years where immigration activists have been standing up in a plane when they wanted to stop the deportation of criminals. Result is usually fines (a criminal liability) and maybe ending up on a no-fly list (which is then not a criminal justice issue).
The passengers in the New Zealand flight were jerks but one of them said:
"I have to say that if watching the safety video is so crucial and you can be escorted off the plane, maybe Air New Zealand should stop making 'Rachel Hunter ice cream ad' safety videos," the passenger said.
in reference to the the way Air New Zealand makes their videos with more an aim of going viral on youtube than getting across a safety message.
Their last video was actually withdrawn because people hated it so much.
edit: removed "criminally" as it seems that this would be a civil case.