Confronting Change

In earlier posts, I’ve discussed working with the Hydra spectrograph on the WIYN Telescope at Kitt Peak National Observatory. It’s a piece of instrumentation that allows astronomers to observe up to 90 objects simultaneously during one telescope pointing. The WIYN telescope can see one square degree of the sky at a time. Hydra has a set of optical “buttons” which can be placed on a metal plate at the position where an object will be in the field of view. These buttons send data to the spectrograph two floors below via fiber optic lines. The device that places these buttons is a robot, which resembles the claw from crane machine. It picks up each button from a parked position and puts it where it belongs.

Sticker on Hydra’s side

This type of multi-object spectrograph is somewhat common now, but Hydra was one of the first of these types of spectrographs built in the early 1990s. In fact, the instrument is older than the WIYN telescope and I helped to commission the instrument in its first version at the Mayall 4-meter. Precision machines like the Hydra gripper suffer a lot of wear and tear in 30 years, so a little over a year ago, Hydra began an upgrade process. The engineers worked with Prod Design and Analysis in El Paso, Texas to upgrade the gripper with new technology. The folks at Prod were struck by how much the gripper resembled a crane machine’s claw and added the sticker you see in this first photo.

Hydra Control Rack

As you might imagine, I’ve become quite familiar with Hydra, working with it off and on for almost thirty years. However, making major changes to the instrument meant learning new procedures. The original Hydra gripper was an analog device that “knew” where buttons were by counting steps using a device called an encoder and then remembering those steps to go back and pick the button up again. If there was some electrical noise or a mechanical problem that caused the count to be less precise, it could miss picking up the button. The upgraded Hydra also uses encoder counts, but it has added new programmable logic controllers and video analysis. It goes to the button, takes a picture, analyzes that pictures and then adjusts it’s position, if needed, to get the button. All this new logic has meant that we have a new control cabinet in the telescope to house electronics. The hope is that this will make positioning more accurate. However, it has meant learning new ways to monitor the process of the gripper at work and recovering the gripper if problems occurred.

This past week was one of the first times I’ve had to use the upgraded Hydra without one of the engineers on hand in case problems occurred. I made sure to review the manual, because it had been several months since my introduction to the upgrades and I reviewed the troubleshooting procedures. The first night of observing started out quite smooth. We had four field setups without any problem whatsoever. Then on the fifth field configuration, a heart-stopping problem occurred. The Hydra control program crashed while the gripper was carrying a button to a new position. Of course, these buttons and their attached fibers are all very delicate and if Hydra forgets the button’s position, there’s a risk that the attached fibers could get tangled. This could prove to be a very expensive problem.

In the old days, a problem like this automatically meant going into the dome, which is a cold proposition on a winter night! Then with a long stick , carefully reaching into the instrument and releasing the button from the gripper jaws, which allow you to open the instrument. After that, you’d have to manually place this fragile, optically sensitive button and fiber back into its stow position. In short, it’s a delicate procedure to do when you’re cold and on a lift in a dark dome!

The new Hydra Handpaddle

Now we have a handpaddle, which lets us talk to the gripper directly. I went to the Troubleshooting guide, refreshed my memory and followed the instructions. For the most part they worked. I was able to control the gripper and set the button down in a controlled way. Unfortunately, we’re still working out some bugs, so I still had to open Hydra and check the button’s real position, but it was a much quicker, safer operation.

Confronting change in procedures that had grown familiar and routine was definitely scary and a little challenging. The things that helped me manage my discomfort were focusing on the familiar parts of the routine, reviewing the new procedures before starting work, and then when a problem did happen, I took a deep breath and used the instructions and my experience to solve the problem the best I could. This past week, the upgraded Hydra moved from a machine that presented me with a little anxiety to a machine I look forward to understanding better.

2 comments on “Confronting Change

  1. The image I grew up with was astronomers looked through a telescope, made notes, then looked through the telescope again. I don’t know if that image was ever actually accurate during my lifetime. But grippers, optical “buttons,” fibers getting tangled, climbing into a dome in the cold to make repairs–none of that was part of my conception.

    I think a lot about how things have changed. Through most of human history, the tools your grandfather was using the day you were born (assuming your grandfather was still alive) would be the same kind of tools your grandchildren would be learning to use the day you died.

    Now it’s, “Gee, kids, it’s Windows 11 now. But I remember back in the old days when I was using Windows 8, way back in…2015.”

    • Yeah, the image you have was pretty outdated by the beginning of the twentieth century. By then, astronomers were starting to capture most of their data on photographic plates. Even the spectra they took were recorded on photographic plates. That changed the game and made a hard, physical record of observations so others could see what the astronomer’s data.

      Another fun misconception I often see in films is astronomers in lab coats. I’ve never worn a lab coat on the job in my life. The only time it might have been necessary was back in those days of photographic plates — to avoid getting chemicals on your clothes. Now in the era of digital photography, it’s rare for us to work with dangerous chemicals. The biggest issue might be oil if you’re working on the telescope drives, and you’re going to want coveralls for that kind of work!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.