If you're using NoScript in the fine-grained manner you describe, and for privacy reasons (not just security), I wonder have you ever looked at uMatrix[0]. Same deal but a bit more performant, and also covers the whitelisting of other privacy-leaking aspects such as cookies, CSS, tracking pixels, iframes, etc.

[0] https://github.com/gorhill/uMatrix/wiki/FAQ