Its going to be easier to operate the charging handle with the weapon cocked. You are not overpowering the hammer spring tension then the disconnector spring tension like you are when charging the weapon from an uncocked position.
It is difficult to assess if you have a weapon problem via this medium.
I liken the effort to charge the weapon from an uncocked position to that of pulling the starting rope on a chainsaw or lawn mower, if that gives you a perspective.
"Many men are able, most aren't willing"