The smell/feel of the cork is never a reliable indicator of a fault nor of a wine's character or quality.
Placing the cork beside you dates from the 18th century when the wineries began branding corks. It allowed the consumers to see that the name written on it matched the name on the label and prevented the restaurateurs from filling an empty bottle with an inferior wine.
You'll be enjoying your wine more after swirling the glass.
Holding the glass by the bowl or top of the stem quickly warms up the wine losing some of its taste qualities.