Sometimes a thing so special needs to be approached slowly, in small steps that get closer over time.

I grew up a few short miles from Los Alamitos Race Course in Garden Grove in Orange County in the 1960s and 1970s. My earliest memories of the modest track—now in the news as the unlikely home of Kentucky Derby winner California Chrome—are seeing it from the back seat of my mother’s car as we drove down Katella Avenue to the Pasty House for Cornish meat pies. Who were the people, I wondered, that frequented the Starting Gate Bar or stayed at Don’s Turf Motel?

When I was around 9, Los Alamitos began to take some hazy shape in my mind, as rumor spread that a family on my street spent their nights at the track. The idea that people left their houses and glowing television sets after dinner was incomprehensible to a young boy who had dinner at home—5:30 sharp every night of the week as soon as Dad pulled into the driveway.

It was years later when I realized what a glorious place Los Alamitos was. At barely 16, my best friend Dave and I would put $3 of regular into his brown Ford Maverick and head to the old Cypress golf course, where you could play 18 holes for less than $10. A few of the back nine holes were behind the barns on the track’s property, and we always got a great view of the infield lake and the grandstand that stood empty during the day. All the action would come under the lights, beginning at 7:30 p.m., when the quarter horses and harness racing began.

Which is what I discovered when I lumbered over legal drinking age. Or maybe I was nearly at drinking age when my friends and I finally took advantage of our proximity to this track and its welcoming attitude toward beer coolers for the quarter horses races. Quarter horses are literally a breed apart—they sprint fast and run shorter than the thoroughbreds, turning their races into seconds-long bursts of excitement. There is nothing quite like it. My friends and I relished, too, the contrast between Los Alamitos and Disneyland, where we were working summer jobs. Every night, they’d scrub the theme park down and repaint it. But the track always smelled like rot and decay.

The Los Alamitos of my youth was a small track, and it’s still a small track—though without the frame of reference of bigger venues, it didn’t seem small to me then. Bets were made, and races won and lost, but what stood out for me were all the characters you’d see at the track. My buddies and I bet for fun; the people around us—mostly men, from all kinds of backgrounds—placed their wagers with the faces of those making life-or-death decisions. Kids roamed around picking up betting slips off the ground to sort through later, looking for a discarded winner. The sights and smells of the place did not fit my cloistered upbringing.

My mother still lives in the house I grew up in, though my childhood friends and I have scattered. But the track is still in me. I’ve dabbled in owning thoroughbreds over the years. And I still go to races with friends, though they’re not the same friends, and it’s not the same track, either; living in L.A., Santa Anita is now the destination. Los Alamitos, always the little track in the O.C., seems even littler.

But give Los Alamitos this much: It’s outlasted Inglewood’s Hollywood Park—originally the horseracing playground of the movie elite—which shut down in December. The fact that Los Alamitos has held out is a testament to its current owner, Dr. Ed Allred, who is also well-known as the co-founder of Family Planning Associates Medical Group, an abortion services provider. Some say that the track will only last as long as Dr. Allred does.

Los Alamitos is currently undergoing millions in renovations needed to accommodate a mile-long track for thoroughbred racing. Perhaps I’ll finally stop by the Starting Gate Bar for a quick one on my way to first post next season.