Troubleshooting

My friend Darla Hallmark sells buttons that say, “The problem with troubleshooting is that trouble often shoots back.” In my job operating telescopes at Kitt Peak National Observatory, I often get to see the truth of that statement. Here I am in my natural habitat at the control station of the WIYN telescope.

My actual title at Kitt Peak is “Senior Observing Associate” and my job is more than being a telescope driver. I see myself as the person whose job it is to make sure the astronomers who use the telescopes actually get the data they hope to obtain. At night, especially at the WIYN telescope, I’m often the only person in the building. It’s quite common for observers using the telescope to control the cameras over the internet and talk to me all night on a Skype connection.

The observatory has a daytime staff of engineers, electricians, mechanics and more. Most of them are tucked snug in their beds when I’m working through the night. So, if something goes wrong, I’m the guy who has to fix it, or find a workaround until the next day when the daytime staff returns to work. I think its a real testament to the design and maintenance of the telescopes at Kitt Peak that serious problems don’t crop up all that often, but when they do, they can be a challenge.

We had one such problem this week at WIYN. We were using the Hydra spectrograph. Instead of an eyepiece or a camera looking directly at the sky, there is a metal plate. Fiber optics in magnetized housings sit on that metal plate and face the sky. A robot within Hydra can move those around so they’re in a position to capture light from distant objects. This week, each fiber was placed to catch light from galaxies approximately 11 billion light years away. As you can imagine, you need to place that fiber in just the right place to catch that tiny bit of light. This is what the inside of the Hydra spectrograph looks like. You can see the fibers on the left-hand side. The robot that moves the fibers is on the right.

The problem we had was that some of these fibers were missing the light. To confuse matters, not all the fibers were missing the light. We saw light from some galaxies. We saw light from all the stars that let us do fine corrections to our pointing on the sky. My first thought was that there was a calculation error and not all the fibers were being placed correctly. The astronomer looking at these galaxies checked and eliminated that possibility. Next, we used a camera on the robot to watch the fibers as they were being moved to see if they were being placed where we put them. The robot did just what it was supposed to do.

The final step in this procedure is that the metal plate on the left gets warped, because the telescope’s focal plane isn’t actually flat. We watched the fibers as the plate was warped. The fibers in the center “jumped.” That’s not supposed to happen. As of this writing, I’m not sure why warping the plate made some fibers jump but not others, but the obvious workaround is not to warp the plate. What this means is that some galaxies will be better focused than others, when we take data, but since we’re taking spectra, that’s not a showstopper. We just care that the light makes it down the fiber. Once that happens, the astronomer can see what elements exist in that galaxy and get information about how far away it is and how fast its moving. As the weeks goes on, that team of engineers and technicians will take the information I learned about the problem and work to find a solution.

If you enjoyed this behind-the-scenes look at my job operating telescopes, you might enjoy my novel, The Astronomer’s Crypt. It tells the story of ghosts, gangsters, astronomers, and a dangerous Apache spirit colliding at a New Mexico observatory on a dark and stormy night. You can learn more about the novel and watch a cool trailer at: http://www.davidleesummers.com/Astronomers-Crypt.html.

If spooky stories aren’t your thing, but you’ll be in Phoenix, Arizona on Thursday, May 9, you can join me at the next meeting of the Phoenix Astronomical Society, where I’ll be talking about the DESI project on the Mayall 4-meter telescope and sharing some behind the scenes photos of the installation. You can get more details about the meeting at: http://www.pasaz.org/index.php?pageid=meetings

The Robots Are Taking Over

I spent this past weekend at Bubonicon in Albuquerque, New Mexico. Yesterday, I moderated a panel called “Are Robots Still Scary? Danger Will Robinson!” As it turns out, I suggested this panel to the event organizers because it occurred to me that I work with a robot quite a bit at Kitt Peak National Observatory and it’s a far cry from the anthropomorphic robots of pulp sci fi and probably more irritating at times than scary. The robot I’ve spent most of my time with is called the Hydra gripper. The reason it can be irritating, is that I’m one of the guys who has to go fix it when it breaks down!

hydra

The gripper is on the right-hand side of the photo above. Its job is to pick up the fibers on the left side and position them on the plate so that they line up with objects on the sky. When the telescope is pointed at the target, light from the objects will go downstairs to a spectrograph, where it will be broken into a spectrum and projected on a camera. Astronomers can take that light and analyze it to understand the chemical composition of the objects they’re studying. These objects can range from stars, to galaxies, to nebulae. Some objects are nearby, others are among the most distant in the universe.

The reason for this post’s title, is that we’re about to get another robotic spectrograph at Kitt Peak. This one is called DESI (which stands for Dark Energy Spectrographic Instrument). Hydra allows us to take spectra of upwards of 200 objects at a time. DESI will let us take spectra of 5000 objects. You can read more about DESI in this press release from the Department of Energy’s Berkeley Lab. The goal of the instrument is to get spectra of all the known galaxies obtainable by the 4-meter in order to understand the phenomenon that’s been dubbed “dark energy.” Along the way, we’ll build an incredible database of spectra available to the entire astronomical community.

This week, I’ll be helping to test a prototype of DESI on the Mayall 4-meter telescope. The fibers of DESI are so closely packed that they aren’t moved around by a system like a gripper. Instead, each fiber is a little robot that turns on its own to optimize its position on the sky. Because of this project and the number of people it takes to get a project like this off the ground, DOE has helped to fund a new, larger control room for the Mayall. You can see the original on the left below. The new control room is on the right.

New-Old Control rooms

As it turns out, ghosts and stories of haunted observatories feature prominently in my forthcoming novel, The Astronomer’s Crypt. Our new control room is one of the “haunted” spaces in the Mayall. It used to be an old lounge and there were two rocking chairs that would sometimes be seen to rock on their own even when no one is in there. It remains to be seen if there are any ghosts, or if this motion was just due to sway of the building. If there are ghosts in the building, I hope they like company, because there’s going to be a lot of it in the coming years!