In the spring
of 2001 the staff of the ARO began the task of replacing the outdated control
system of the HHT. The old antenna control system was based on a single VAX
computer interfaced to the antenna control hardware and spectral line backends
mainly through CAMAC electronics packages. The CAMAC electronics was
obsolete, difficult to maintain, and was increasingly unreliable. The VAX
control computer was also dated, but couldn't be replaced since the CAMAC
electronics interface requires a Q-bus machine (which is no longer made).
When the old
control system was first written, the computing power in microprocessors was
very limited. So instead of doing all time-critical computing in a dedicated
microprocessor, only the antenna servo loop was done in the microprocessor.
The other time critical calculations were done in programs running on the VAX,
which in the old design, had to monitor and update the microprocessor commands
four times a second.
was the VMS operating system. Since it was no longer very popular, many of
the current data analysis software packages either aren't available under VMS
or a VMS implementation of their latest versions was not available. Thus,
often the data analysis was either done with older version of these packages
or the data had to be moved to a UNIX platform for analysis.
goal was to completely replace the old CAMAC/VAX control system with modern
hardware and utilize State of the Art operating system technology. We modeled
the new control system after the one developed by the National Radio Astronomy
Observatory for use on the Kitt Peak 12 Meter Radio Telescope. Most critical
systems are Linux base x86 hardware. This allows us the most flexibility in
hardware choices and prevents us from getting 'locked' into one vendor’s
wares. The following is a brief description of the current 12 Meter system:
It was our goal
to apply the same control system philosophy to the HHT.
Instead of a command line control interface, the new control system is
completely graphical in nature. All operator and observer interfaces are
point and click. In addition all system and sub-system parameters are
displayed via graphs and plots.
Remote Observing. --
Remote Observing Manual.
control system had remote observing built in from the start. It is modeled
after the highly evolved 12 Meter remote observing package. The following is
a description of the 12 Meter remote system which is fully applicable to the
current SMT remote package:
telescope has made significant advances in the development and
implementation of remote observing. An astronomer can log-on to the
telescope control system from virtually anywhere in the world and set up a
software package in a matter of seconds that entirely mimics the system at
the telescope. The astronomer views the data on-line in real time as if they
were at the site. This capability is augmented by highly skilled telescope
technicians on the site who perform all necessary local functions for the
observer, such as tuning the receiver to a new frequency. The observer
interacts with the site technician via a "chat window". Even a laptop
computer can be used for remote observing. A unique software/hardware
package has been developed over the years for this purpose, and has been
thoroughly tested by observers from all over the world who use these remote
capabilities. Moreover, it allows for extreme flexibility in telescope
schedule, and is perfect for long-term monitoring program and/or sudden
developments, such as the appearance of a new comet.
Debugging. One of the more useful features of a highly evolved
remote observing package is in it's utilization during remote debugging.
Because of the 'you are there' nature of this package, technicians and
engineers are able to diagnose most problems faster and prevent the majority
of unnecessary trips to the site. Thus, improving telescope efficiency
training. Student astronomers can be trained in mm observing by
utilizing the SMT remote observing package. Without the expense of travel,
astronomy professors can train a multiple number students in all aspects of
radio astronomy observing with out leaving their home institution. Veteran
astronomers can observe at the site while their students look in from back
home and vice versa.
Collaborators. Large collaborative projects can be carried out with
only a few if any astronomers actually traveling to the site while all
others can observe remotely. Each remote observer can share different times
to oversee the science while the others are teaching, sleeping or working
on other projects.
Monitoring. Another feature of remote observing is utilizing it for
remote monitoring. By setting up a permanent remote session in the downtown
offices, the Tucson staff can 'look in' on the current observing and thereby
oversee the facility. More then once a situation has been discovered
remotely on the 12 Meter and corrected by the Tucson engineers.
new control system is of a distributed design, the processing power is
magnified many times over. This also lends itself to parallel processing.
Many different tasks are carried out simultaneously. This design allows the
backend computers to concentrate strictly on the process of data taking.
Tasks such as graphical displays, file servers and on-line analysis are
handled by separate processes in other computers. So for instance, all the
while the file server is writing the just completed scan to disk, the
on-line analysis is reducing it and the graphics engine is displaying the
results, the backend is already taking the next scan. In fact, while
observing, the backend never stops.
Modern Independent Hardware.
One of the
problems with the old system was its complete dependents on proprietary
hardware and operating systems that are difficult to maintain and in most
cases no longer manufactured. Our goal was to design the new system around
off the shelf hardware that is currently manufactured by many different
vendors. The operating systems are of open system design.
By the very
nature of our distributed design, hardware can be selected for each specific
task. By utilizing generic off the shelf hardware we are insuring ourselves
future expandability and modernization. Due to our modular design, a single
functional peace of hardware can be upgraded without affecting the other
much inefficiency in the old system. Spectral line beam switching had an
inherent inefficiency that ranged from 25 to 50%. This was primarily due to
the non synchronized nature of the hardware. Also large periods of dead time
existed at the beginning of and in between scans. Both of these and other
minor inefficiencies were addressed during the upgrade. We have achieved a
In the old
SMT control system, data could only be dumped from the backends at a rate of
once every two seconds. This limited the speed at which a On-The-Fly map
could be acquired. The new control system dumps it's data at the rate of 10
times/second. This will allows for OTF maps of the same size to be taken at
1/20 the time.
Capability with 12 Meter.
SMT control system looks and behaves much the same as the 12 Meter system,
several advantages can be realized:
Observer Familiarity. Observers will see the same computer
interfaces both on site and remotely. This reduces the observer's learning
time and simplifies observing manuals and procedures.
Operator Familiarity. Operators also see the same interface at both
sites. This too simplifies operator training and facilitate the use of
operators working both sites. New operators can be trained on the 12 Meter
and then moved to the SMT with minimum retraining.
Troubleshooting. Both sites utilizing essentially the same control
system, will simplify diagnosing problems as technicians and programmers
need only know one system
The new control system has been
in place and operational since November 2002 and since then has undergone
steady improvements. All spectral line back ends have been implemented and
observing procedures have been finalized. We continue working to improve
observing efficiency. Also, we are working on more graphical user interfaces
to afford the operators and observers more information in real time to allow
informed, fast decision making in regards to their observational project.
See figure 1 in the section on OTF mapping for an example of fast
mapping at the HHT using the new control system.