jQuery is 32 Kb compressed and minified.
Pretty small, really. I’d say heck, a hero image might be 10× that size alone!, but as we know, all bytes are not equal in web performance. JavaScript needs to be parsed and executed, and might even be a blocking resource depending on where and how it’s loaded.
So is it worth it to refactor code and remove jQuery from a code base?
GOV.UK did exactly that, a completely isolated change, and measured the results.
… on a low specification device and 2G mobile connection …
the time it took for the page to completely draw the pixels to the screen (visually complete) dropped from 11.3 seconds to 9.4 seconds (a 17% improvement).
A bigger change that I would have guessed for that one thing alone.
They even went so far as to publish a document on how to remove jQuery, which is a lovely example of doing everything you can to improve things.