At Home With the Revered Chef Peter Kelly

“Try this” proffers my dinner host, pouring a crystalline liquid into three shot glasses. In general, I do not partake of spirits unescorted by a mixer, but this vodka was exceptional – smooth, clean, and warm on the palate. No sooner has its elevating effects taken hold than there appears a bottle of Champagne, 2006 Veuve Clicquot, one of the best. As we celebrate nothing in particular I notice, on the kitchen counter, some sake, along with a bottle of rosé wine (Domaines Ott, 2014), and a Napa Valley red (1992 ™Silenus∫). Out of view, poised for after-dinner deployment, was a 20-year-old tawny Port (Porto Kopke). Considering that my generous host and I share deep Irish roots, this impressive roster seems appropriate with the nine-course, mid-winter dinner to come. I am at the Blauvelt home of Peter Kelly, a spacious, open, contemporary structure in a heavily wooded section of town. He moved here with his wife, Rica, in 1995.

Kelly is a revered chef who is to Hudson Valley cuisine what Washington Irving was to the Catskills. This happens to be a Monday night, chef-chills-out night. As with many 70-hour-a-week restaurateurs, this is Kelly’s time to relax and reacquaint himself with Rica, his partner of 25 years, and their 21-year-old son, Dylan. 

  “Mondays were always a holiday for us,” Dylan remarks, standing at an eight-foot-long, marble-topped island. A senior at Cornell University, The SC Johnson College of Business, which has a Hotel Administration component, he professes little interest in professional cooking, despite having worked in his father’s establishments from age 14. He will graduate this May with a degree in Hotel Administration. Born one year after his parents moved into the house, he attended Tappan Zee High School, where he played varsity basketball and was president of his senior class. 

As an appetizer, dad sets out creamy burrata with rosy prosciutto di Parma, a dab of whipped ricotta with local honey, croutons, and pristine littleneck clams served warm with bacon and shallots.

Kelly is a poster image of a chef: a broad, genial expression, reddish-gray hair, an ample diameter, and the ruddy complexion of someone who spends a lot of time near gas flames. He is widely credited with introducing sophisticated new American cuisine to the lower Valley — and by extension the entire Valley — and has mentored countless young culinarians far and wide. Until selling two of his establishments last year, his company, Xaviars Restaurant Group, comprised Xaviars at Piermont and the adjacent Freelance Cafe and Wine Bar (both sold last year); Restaurant X and Bully Boy Bar, in Congers; Events by Xaviars, a catering company; and X2O Xaviars on the Hudson, in Yonkers.

 “I have so much respect for Peter and what he has done,” says David DiBari, chef/owner of the acclaimed The Cookery, and The Parlor, both in Dobbs Ferry. “He was always there for me when I was young and stupid. And he made it okay for other new chefs in the Valley to do their thing.” 

For a journalist, interviewing chefs about Kelly can get a little repetitive, with nearly everyone lauding his talent and, above all, his professional generosity.    

Our following course was Alaskan king crab with chive butter. Alaskan king crab is something I rarely order because virtually all of it comes in frozen. This was among the best I had ever tried, and so rich that at first I thought it was lobster. Served with it were delicious King oyster mushrooms with chive butter, served with the rosé, carrying a faint sweetness that goes well with seafood. 

When Kelly came on the scene, in the early ’80s, the definition of fine dining in the Valley was French dining: La Crémaillère, La Panetière, Auberge Maxime, Buffet de la Gare — elegant, exclusive, and expensive. American-style establishments were, for the most part, homespun, and Italian fare was Neapolitan, meaning red sauce. The Culinary Institute of America, which moved from New Haven, CT, to Hyde Park in 1972, had yet to pollinate the landscape with young talent.

Kelly’s Runyonesque biography would make an inspiring feel-good movie. Tenth in a family of 12 children, the family experienced severe financial struggles. For a time they lived in the Schloebaum Housing Project in Yonkers.

“We were all at home with our parents,” he recalls. The family ate at a picnic table covered with a slab of Formica. When Kelly was 12, his father, an insurance man, passed away. The family had to further pool its resources, with the kids delivering newspapers, shoveling snow, washing dishes, mowing lawns.

The family relocated briefly to Croton-on-Hudson, then to Wappingers Falls, where, at 14, Kelly found work washing dishes at an old-fashioned German restaurant called the Forest House. Two years later, while still in high school, where he was on the wrestling team, he talked his way into becoming a banquet waiter at Dutchess Manor, a continental-style restaurant in Beacon. (He managed to remain in school, but that was the end of wrestling.).

To be sure, Kelly’s formidable work ethic partly derives from the deprivations of his childhood. I asked how he could possibly, as a teenager, hold down what amounted to two full-time jobs.   

“You do what you have to do,” he shrugs. “I think it also has a lot to do with the Catholic faith that my mother instilled in us.” Following graduation, while still working at Dutchess Manor, he enrolled in Dutchess County Community College, in Poughkeepsie.

Later that year, he happened upon a new restaurant in Cold Spring by the name of Plumbush. It was a charming structure — a rambling 19th-century Victorian mansion with fireplaces in every dining room — owned by two meticulous Swiss restaurateurs recently of the renowned Stonehenge, in Ridgefield, CT.  Kelly’s earnest and professional demeanor, along with his advanced dining room skills for someone his age, got him hired on the spot. The menu was Swiss continental — truite bleu (trout poached in bouillon), crêpes à la Suisse, shrimp in beer batter — and the restaurant was soon anointed with three stars by the New York Times (“a culinary Mecca”) and raved about in the regional press.

At this time the food scene in Manhattan was detonating with creativity.  Many young Americans — a good number from the Culinary Institute — who had trained in France were returning to the States and applying European techniques to American ingredients, much of it from the Valley. It was labeled New American cuisine.

“I knew I had to go down there,” Kelly recalls.

After knocking on the doors of many of the city’s most prestigious restaurants, he landed a spot at Laurent, an ultra-formal French establishment frequented by the likes of Jackie and Aristotle Onassis and the high-fashion crowd. Its illustrious bar, with original works by Salvador Dali adorning the walls, was one of Midtown’s most stylish redoubts. Kelly toiled for a year at Laurent, further refining his front-of-the-house skills. At the end of nightly service, he hustled to Grand Central Terminal to catch the 12:20 a.m. train to Croton, followed by a half-hour drive home. He was due back at the restaurant by 11 a.m.  

By 1982 Kelly felt he had earned his hospitality degree. Now it was time for graduate school. “I needed to go to the great restaurants of France to see what the fuss was all about,” he says. He called on many Michelin three-star establishments, among them L’Auberge du Pont de Collonges (Paul Bocuse), Taillevent, Troisgros, and Restaurant Guy Savoy. 

In 1983, he was offered the opportunity to take over the restaurant at the Highlands Country Club in Garrison, which he called Xaviars. The setting was cheerful and unpretentious, and the food contemporary American, highlighting Hudson Valley provender. All was going swimmingly until several days before opening, when Kelly and the chef fell into a heated argument over — and I am not making this up — whether beer batter should contain eggs. Kelly insisted no. The chef quit. Faster than you can fry a battered shrimp, dining room maître d’hôtel Peter Kelly became Peter Kelly, chef de cuisine. His girlfriend at the time became the maître d’hôtel.Among other things, the trip disabused him of the conception that the French, particularly French waiters, considered Americans one hereditary step from eating out of a trough. “They treated me like royalty, and I didn’t speak a word of French,” he recalls. Upon returning home he gathered a number of classic cookbooks to learn more about what he had tasted — Jacques Pepin’s La Technique was one of the most influential. Kelly still considered himself a dining room professional but, he reasoned, the more expertise he could garner, the better.

With no experienced cook, no wine steward (that was to be Kelly), no financial reserve, and no advance publicity, Xaviars opened its doors. Was he nervous? “I looked at it this way. I was only 23, and if it failed, I’d pick myself up and do something else.”

Then he added, “Besides, it was better than being at home — at least I got to eat.”

It did not fail. Within months, Xaviars in Garrison garnered a rating of 29 out of 30 from the Zagat Survey (the highest such rating to date); shortly thereafter, the New York Times assigned three stars, and there were raves from regional publications. The restaurant enjoyed a 20-year run. 

 “One of the things that made Peter so great in those days,” says Glenn Vogt, owner/partner of RiverMarket Bar & Kitchen in Tarrytown, and Crabtree’s Kittle House Restaurant & Inn in Chappaqua, “is that because he had spent so much time in dining rooms, when he was a chef he had a great feeling for customers.” Even when he became a “celebrity chef,” colleagues observe, he remained grounded and always placed customer satisfaction over ego. 

Adds Vogt, who worked with Kelly on a number of projects over the past 30 years: “Some chefs, if you ask for a well-done steak, they say no. With Peter, if you want a peanut butter and jelly sandwich he’ll do it, and it will be the best peanut butter and jelly sandwich you’ve ever had.”

Four years after opening Xaviars in Garrison, Kelly came across a weathered clapboard building in the scruffy waterfront town of Piermont. One half served as a 24-hour tavern for shift workers at local factories. By this time the area’s two paper mills had shut down, and smaller enterprises followed. Piermont was taking on water. 

His creation, called Xaviars at Piermont, was something the lower Valley had never seen. It was a diminutive space, just 12 tables, elegant and formal, with captains in black tie. (His brother Ned oversees service in all of the restaurants). There were crystal chandeliers, Versace china, Baccarat figurines, and walls adorned with vibrant paintings. On the menu were then-exotic creations like salmon with Chinese mustard and honey, Dublin Bay prawns with basil, and filet mignon with black pepper and golden raisins. Word soon spread about this most improbable, though expensive, new neighbor. The adulatory food press swarmed in, and it became a destination for the lower Valley. 

Shortly after opening, Kelly attended a function put on by American Express.  There he met a woman who worked for the company in new client acquisition. “There were sparks,” recounted Rica as our home-cooked feast progressed with a spectacular roasted rack of lamb with rosemary jus, and roasted cauliflower with lime and sweet-salty nuoc cham. Soon thereafter she rang him up, with the proximate purpose of pitching an American Express account for the restaurant. Thank you, he said, but we take cash only. Subsequent efforts proved unavailing.

 “He said no, and was a real holdout,” she says. “I thought to myself, this might not be the best companion for me, work-wise.” 

Kelly held out; Rica did not. They began dating, and married in 1993. 

  Two years after opening Xaviars at Piermont, Kelly decided to create something more casual that would be accessible to local residents. Shortly after opening Freelance Café & Wine Bar, the media devoured it, including another three stars from the New York Times.      

In addition to taking on the mantle of “celebrity chef,” both in the Valley and nationwide (owing to appearances on The Food Network and a James Beard nomination), he is also involved in countless charities and benevolent causes. 

“You have to give back,” he says. One of the most ambitious is called the Chef Peter X. Kelly Teaching Kitchen, at New York-Presbyterian Hospital in Cortlandt Manor. Courses on healthful cooking are held for children and adults in a gleaming professional space, many involving physicians in different specialties.  

One might surmise that Kelly, as an entreprenuer, is one of those golden touch types, untainted by misfortune.  In fact, he has experienced more than one crash landing. The first was a company called Impromptu Gourmet — delivered meal kits for assembly at home — launched in 2001 with some of the most famous chefs in the country. 

“It blew up in my face,” he huffs.

It was an idea a dozen years ahead of its time. Today, these services are blossoming nationwide. 

In 1997, Kelly looked nine miles north to the hamlet of Congers to open his most enchanting venue, Restaurant X and Bully Boy Bar (yes, all of these X’s can be confusing, as his middle name is Xaviar). Nestled on several sylvan acres with a large pond, the restaurant has five dining rooms, each with a different motif, from traditional American to glass-fronted modern. The Zagat Survey rates it 4.4 out of five, calling it “impeccable” and “expensive.”

In 2000 Kelly took the biggest risk of his career. The city of Yonkers, after years of over-promising and false starts, was moving toward redeveloping its barren and unappetizing waterfront as a residential and commercial hub. In the middle of it all, towering over the shoreline, was a huge, scabrous, three-story, turn-of-the-century iron pier — an eyesore if there ever was one. Announcing that he wanted to create something spectacular in his hometown — a city where he was once so needy that shoes were a luxury item — Kelly struck a deal to build a dramatic restaurant perched over the Hudson.

Construction was delayed for two years as a result of 9/11. Xaviars X2O on the Hudson opened in 2007.  Sheathed in glass, and with a 25-foot-vaulted ceiling, diners are afforded sweeping views of the Palisades to the west and glittering Manhattan to the south.  Kelly’s vibrant, ecumenical style of cooking — weaving American, French, Spanish, Italian, and Asian (with a sushi counter in the bar) —  garnered widespread acclaim. 

Our final conversation is at the sleek sushi bar at X2O. Upon being seated, we are presented with long test tubes planted in crushed ice — Slovenia vodka (the same vodka he served at our first meeting). Kelly promotes the vodka with the considerable assistance of friends Bill Murray and Mikhail Baryshnikov.

The dinner at Kelly’s home concludes with the sweet and nuanced Port, a fine foil for our last course, a citric lemon tart. I ask the chef if he looked forward to having only two restaurants for the first time in many years.

“I’m happy because I can now spend more time with customers rather than be racing all over the place.” 

 And will this afford more lazy leisure time at home?

“Probably not.”