Popular theory: Keeping your nose to the grindstone means to work diligently. A grindstone is a round rotating stone used for sharpening knives. It is said that sharpening knives for a living using a grindstone was extremely difficult work. The knife sharpeners were prone to keeping their faces very close to the wheel, in order to insure that the blades were being properly honed. This hard work and attention to detail on their part is generally thought to be the origin of the phrase.
Alternative theory: I was able to uncover this whilst perusing some back issues of The Journal of the Medieval Curing Arts, while waiting to purchase a Power Ball ticket at our local convenience store. Some Middle Ages (though not necessarily middle-aged) physicians were feeling guilty about utilizing their skills only on behalf of the wealthy. At the annual meeting of members of the Physicians, Apothecaries and Taxidermists Guild in 1247, a certain Dr. Phlegm Sputum addressed the gathering. “Why should not the common folk, especially those afflicted with unusually large, unwieldy proboscises, be made to look as attractive as the nobles to whose every whim we cater? We may not be able to do anything about their wretched clothing, appalling lack of manners, or vomit-inducing odor, but surely we can throw in the occasional nose job.” There was much applause. Knowing that their wealthy clientele would never allow themselves to be exposed to a medical instrument that they surmised to have been within 50 paces of a pauper, the enterprising physicians enlisted the help of the Blade Sharpeners Guild, who donated some dull knives and a few used grindstones. Soon thereafter, many a bulbous-beaked peasant could be heard whimpering, if not screaming, while their benevolent medical provider exclaimed: “Dammit, keep your nose to the grindstone! I have an 11:30 tee time.” This seems to me to be a very plausible origin of this expression. I do so hope you agree.