The Case for a Logging Standard, Part 2

In my previous post, I stated several arguments for a standard file format for holding astronomical observations. The arguments fell into 3 categories: data preservation, flexible access to and presentation of the data. This time, I'll describe a candidate that solves many of these requirements that is available now. Better yet, it is open source and free.

A year or so ago I stumbled upon something that really solved the aforementioned requirements well - an open source XML based standard for recording observations called COMAST. The advantages of recording observations in an XML format are that the content of an observation can be defined clearly for any interested party (program) to produce or consume. XML also lends itself well to reformatting into different file structures or visual presentation. All of this presents in a platform independent context. If you are keeping tally, that's open source, free of charge, platform independent, flexible access and flexible presentation. Hmmm, cool.

The COMAST effort is led by a small number of developers from Germany and Belgium. Fortunately for me, there is English documentation (my German is pathetic.) If you would like to read about the COMAST XML Schema, you can visit the web site at http://observation.sourceforge.net/en/index.html Beware if you aren't an XML techie.

After studying COMAST for some time, I decided that it described observations well enough and generally enough to be a very good candidate solution to my perceived requirements. I decided to put it to the acid test - try to import and export observations from Deep-Sky Planner using the COMAST format. After some discussion with the founding designers of COMAST, I have succeeded in transferring COMAST formatted data into and out of Deep-Sky Planner. There are idiosyncracies, but it works. Double cool.

COMAST does a thorough job of describing your observing site, equipment, time of observation and visual results. It can currently accommodate imaging equipment and results, but not as thoroughly as I would like. Deep-Sky Planner continues to provide storage and reporting of imaging observations. I suspect that the COMAST standard will catch up and I hope to take part in that effort.

COMAST does include some data that is more relevant to German observers, like one's Deep-Sky Liste (DSL) ID number and a DSL object rating code. I'm sure that observers elsewhere have familiar rating systems, association IDs and the like. These items reveal the roots of the COMAST project but they aren't required. They probably should migrate into an extension for German observers from the basic standard, but for now they can be ignored if desired. The nice thing about the COMAST design is that it makes extensions possible with predictable results. For example, should there be an extension developed that describes imaging observations, older software could accept and produce legal files with no changes - it would simply not support imaging extensions.

So what can COMAST do for you? At present, it can provide you with a observation data migration path between a couple of software products. Should that number increase, we may just have ourselves a viable world standard for sharing and preserving observations. Very cool indeed.

The Case for a Logging Standard

Logging astronomical observations is a very personal thing, and not all observers keep a log. Perhaps visual observers are the least likely to record observations but many do. Most imagers at least keep records of their exposures, and the end product - an image - serves as excellent documentation. Sketchers probably keep the best records of all since their work is, by definition, recording detail manually. With observers' logging habits so varied, why do we want or need a standard?

From an observer's viewpoint, it may be less than apparent. Many of us record observations so that we can refer back to them and determine whether we've seen something before, or how it looked in the past, or when and where we saw it. Others collect observations and present them for an award. Still others want to publish particular observations on a website or blog. This really defines 3 different usages: searching, managing and reporting. There are, of course, many ways to meet these use cases: using pen and paper, a word processor, a spreadsheet or a database are the usual suspects. Any of these work well as long as we want to stay within the same process - a closed system in the algebraic sense.

So what if we want to venture forth from our tried and true logging practice? Perhaps we want to contribute an observation to a scientific collection (e.g., AAVSO), or publish them onto the web, or enter them into a program that offers a new, desirable capability? These cases require re-entry of the logged data to some extent, a tedious process at best. Worse yet, what if your log becomes inaccessible? Think of a hard drive crash, lost installation disc and a program that is no longer supported. Yuck.

Enter a standard file format for observing logs. The first three cases mentioned above are likely to be solved by having a standard because the more common the use case, the more likely that someone can, or will, solve the problem at least once. As a developer, I would be happy to support one exchange format, but supporting one for every file format requested by users is hard to justify.

The disaster scenario described above is perhaps the best reason for users to demand a standard log file format. I like to think of it as a warranty for the observation data I have spent years accumulating. If a program I have used to record observations either stops being updated or becomes less appealing, at least my data can move forward with me via a standard format file.

Developers will only support a standard if it is a marketable feature. That means that users will have to demand it as a must-have feature for a standard to gain the traction that makes one ubiquitous.

I wonder how many more observers would record their observations if they could enter them once and be assured that they wouldn't have to re-enter them time and time again. I also wonder how many scientifically valuable observations are lost because they can't get out into the astronomy community.

Until there is a standard available for users to demand and for developers to support, the arguments above are just conversation material. Fortunately there is a candidate under development. More on that next time.

Updated SQM-LE Reader

In my last post, I mentioned the imminent release of a new model of Sky Quality Meter. Unihedron released the new product on time and it is shipping now.

Since that post I've had some time to use the meter with SQM-LE Reader software. One of the first questions a friend asked was 'what does the reading really mean?' He was looking for a way to relate the reading in magnitudes per square arcsecond to something more familiar - like visual limiting magnitude. Fortunately, the Unihedron website has a link to such a computation.

It became apparent quickly that SQM-LE Reader should include this computation along with other reading data. Furthermore, the ability to write readings to a comma separated value file was needed; whence, SQM-LE Reader v1.1. The CSV file format is easily imported into a spreadsheet for further analysis, or browsed with a text file utility such as notepad. These capabilities will appear in a future release of Deep-Sky Planner, but in the meantime they are available for free in SQM-LE Reader.

With these tools in hand, I hope to quantify the darkness of observing sites over the span of an evening, and over the span of seasons. I'd also like to determine whether the visual limiting magnitude computation is accurate (at least for my eyes.) I look forward to collecting and analyzing data over the coming months.

Finally, I've compared readings from the new model with those taken simultaneously with an SQM-L model device. The readings are consistently within .01 of each other, well within the stated accuracy of the device. How about that - delivered on time and functioning to specification!

Pacific Astronomy & Telescope Show

The First Annual Pacific Astronomy and Telescope Show (PATS) was held in Pasadena, CA on Sep 13-14. There were a few new wares to be seen which was exciting and fun. The long trip from North Carolina to California, coupled with intermittent Internet service at our hotel, has resulted in a delay in posting this report. Having returned to the office now, here are some highlights of the show.




Saturday was the busier day by far and included the big announcement from Tele Vue of yet more Ethos eyepiece models. Knightware joined the new product fray by announcing SQM-LE Reader, a free program that can be used to read and display data from Unihedron's new Sky Quality Meter LE. The new device is scheduled to begin shipping at the end of this month but the software is available now (http://knightware.biz/sqmreader.htm). To get an idea of how the meter looks, note in the photo below the small black box in front of yours truly, attached to the yellow cable. That's an engineering sample but the shipped version will look the same.

We had the pleasure of talking to many, many interested attendees and met some long-time users of Deep-Sky Planner. It's wonderful to put faces to names. In addition to attendees, we spoke with several product engineers, software developers and store owners. It's always good to get a read on the astronomy product marketplace from the perspective of suppliers. Further, we had some detailed discussions of ideas for new features for Deep-Sky Planner. Explaining the software's capabilities now, and exploring new ideas for it in the future are helpful ways of defining the product.


This show has all the promise of becoming an annual destination for amateur astronomers on the west coast. The organizers obviously did a huge amount of work bringing this show together.

All photos by Mark Lang

Command & Control

Knightware has been working recently on providing support for a new partner's upcoming product. While the official announcement will have to wait a bit longer, I can divulge that this bit of work has allowed me to return to my roots - back into the realm of data acquisition and control work.

Maybe it's a summertime thing...
The last data acquisition and control work that came through the office was about 15 months ago when support for DSC and telescope control was added to Deep-Sky Planner. Testing different ASCOM driver implementations was a reminder of how differently device manufacturers design their products' communication and control capabilities. Synchronous telescope control? Yuck.

For most of the first 20 years of my career, I worked on data acquisition and control with various devices ranging from the US Navy's NavStar satellite navigation system (a predecessor to today's GPS), to F-14 air combat simulators, to electronic power meters. Fortunately, the 'control' part of my experience applied only to power meters. Power usage and substation relays are not as exciting (grin) as F-14 weapons systems and navigation satellites.

Although working with telescope control was fun, testing this upcoming product has been a joy. Communications are very clean and stable. Torture testing has been a real disappointment - no big hang ups even with Vista. A little more testing and we'll call this a wrap.

More to come!
Please tune in to this blog Sep 13-14 or visit the Knightware booth at the Pacific Astronomy and Telescope Show. We'll be talking about Deep-Sky Planner and putting it through its paces, and we plan to announce support for the mystery product too.

Apollo's Legacy

When Apollo 11 landed successfully on the moon 39 years ago today, my life changed. I was allowed to stay up past bedtime that night to watch the murky pictures of Neil Armstrong stepping onto the lunar surface and uttering those now famous words. He chose them well as it was truly 'a giant leap for mankind.' Watching the events unfold reaffirmed my fascination with science and space exploration. At that point in my life, I was sure that I would be the first person to set foot on Mars. I took another road more traveled, and NASA lost focus on that very difficult goal, but the impact of that successful moon landing has never left me. Even now when I see that video from the moon I get goosebumps and a tear, emanating from several things: relief that the astronauts came back safely, a deep appreciation for the science and engineering that was required and pride in America.

I wonder how many of today's scientists were pushed into their fields by the success of Apollo, and what the value of their contributions to mankind might be. Maybe the Apollo missions were a bargain in terms of money. If you think of the role that technology plays in society now, it's hard to imagine where we would be if technology had not developed quite to the point that it has.

A couple of nights ago, I watched a particularly favorable pass of the ISS with my son and husband. As husband took photos, my son asked whether the ISS would be a resupply station for missions to the moon. While I didn't say it, I was thinking that my teenage son's generation just might be affected by a return to the moon as mine was nearly 40 years ago. I hope that it works out that way, and that society can benefit from another burst in technological innovation. Maybe ISS will resupply more than just materiel.

Photo courtesy of NASA

What to Call Them?

As far as Deep-Sky Planner is concerned, minor planets shall henceforth be called asteroids ...

While implementing support for minor planets in late 2006 for Deep-Sky Planner, I had a real dilemma as to whether these objects should be called 'minor planets' or 'asteroids'. The IAU meeting in Prague the previous summer had just brought the term 'dwarf planet' onto the world stage, and demoted Pluto to some mysterious status that still befuddles me. The term 'minor planet' seemed a little like damaged goods.

On the other hand, the Minor Planet Center was the purveyor of orbital elements for these beasts, so the term 'minor planet' must have been the best moniker. So 'minor planet' stuck.

In the November 2006 issue of Sky & Telescope, Editor In Chief Rick Fienberg wrote about the IAU shenanigans in his Spectrum piece. I read it and agreed with his argument against redefining Pluto and the confusing term 'dwarf planet'. At that time I thought about asking Rick about the term 'asteroid' as opposed to 'minor planet', but I pressed on with development, and so it was.

Fast forward to 2008, and a bit of serendipity. I ran into Dr. Fienberg at NEAF in April. I asked him about my dilemma and whether he still felt as he did when he wrote that Spectrum piece in 2006. His feelings hadn't changed, and he liked 'asteroid'. That clinched my decision to make the change to 'asteroid'.

As for Pluto, it never got demoted in Deep-Sky Planner. I hope the IAU reinstates its planet-hood when they next meet in 2009. That seems befitting for the International Year of Astronomy.

Unit Tests & XP

Back from the fun at NEAF, I'm back to the usual routine. Lately I've been making some fundamental low-level changes so I'm writing unit tests to keep things under control. Following the eXtreme Programming paradigm entirely really doesn't make sense to me, but using it in select areas does.

Case in point: I have a class for an equatorial coordinate position, including computing star atlas chart cross-references. Writing tests that make sure the correct chart reference is computed is pretty straightforward and allows testing along chart boundaries. On the other hand, the class has a couple dozen methods so writing tests for the really mundane methods (e.g., copy constructors) is a little torturous. Further, some of these methods have been around for north of a dozen years, so I have a good comfort level with them.

While at NEAF, I mentioned unit testing to Steve Bisque. He said he doesn't use them. I told him they can take as much time to write as the code under test. I think that sealed the deal for him - no time for all that. Ultimately we as programmers need to be aware of unit testing and what it can do for our products, but the extent of their use really depends on schedules and other testing procedures in place for the product. I suspect that Software Bisque has a pretty solid system test procedure, so I guess they cover QA that way.

For me, using unit testing judiciously is the way to go. It gives some assurance for the low level stuff while system testing addresses the higher level. System tests are presently done with a combination of manual testing, Python scripts, and a test team (in that order.)

PS, in the true story category:
I once interviewed a young man for a programming position with my employer. The interview process placed the interviewee at a conference table with 3 staff programmers for what we called "the inquisition". The guy told us that he had never written code with a bug in it. There was a pause as we shot glances of astonishment around the table, and we politely continued the interview for about 20 more minutes. The guy had absolutely no chance of getting the job after that. If you've ever written a program, I bet you're laughing - loudly...

NEAF 2008 - Post Mortem

The show was really a lot of fun. I've returned to the office with a pile of business cards and a sheet full of notes taken while discussing additional features for the product. My head is full of ideas - it's a good thing I wrote them down!

I will continue the blog - it has been received well and it gives me a chance to communicate a little more informally.


A Special Note to Friends and Customers in Southeastern Virginia
We drove on I-95 through the Richmond-Petersburg corridor yesterday and wondered why there were SO many accidents, including a HAZMAT team addressing an overturned truck. Now we know that tornadoes ravaged the area. Please know that you are in my thoughts as you recover from the damage.

NEAF 2008 - Day 2

Day 2 at NEAF brought a lighter crowd, more speakers and more new and existing customers.

Today I had the good fortune of speaking to many media people - Charlie from Amateur Astronomy, both Parkersons from Astronomy Technology Today and Tom Trusock from CloudyNights. I enjoyed discussing the show and their particular corner of the astronomy world. All of these gentlemen bring something special to astronomy. If you haven't checked out their part of the world, do so!

A real highlight of the day came when I met Roger Sinnott of Sky & Telescope. I tried to catch him several times at the S&T booth, but always missed him. The word got out and by mid afternoon he came and found me. Roger helped with Deep-Sky Planner questions as far back as 1993. I've always enjoyed our correspondence but had never had the opportunity to meet him in person. We share an interest in the mathematical side of astronomy and of course his stellar cartography skills are well known. He is a gentleman indeed.

Other editors from Sky & Telescope visited as well, including Rick Fienberg. My business dealings with Rick go back to about 1995 - longer than it seems possible. As with Roger, Rick is a gentleman who has played a vital role in astronomy. It was a pleasure talking with him.

The show was a huge success - both from a business standpoint and from a consumer standpoint. I have befriended vendors in neighboring booths, learned about a bunch of wonderful new products (I have my eye on the new SolaREDi telescope from Daystar), and discussed potential uses for Deep-Sky Planner that sound very useful. This has been a long weekend but a very rewarding one. Now to get back into the office and work on these good ideas...

Bill Burgess of Burgess Optical was in the neighboring booth. He might have been the most animated vendor in the building! He and Tammy were lovely folks from Tennessee so surely our paths will cross again soon.

Photos by Mark Lang

NEAF 2008 - Day 1

Day 1 started with some excitement at the Tele Vue booth. If you haven't heard yet, the 8 mm Ethos has been announced. The photo above shows the announcement being made just before the show opening.

The Knightware booth was busy as we met some existing customers and many new ones. Friends and business associates stopped by making for a fun day, and seeing people that I haven't seen in years was particularly gratifying. Our putting faces to names continues to be a highlight of the show. We have fielded lots of questions and some interesting uses for the software. Creative ideas really seem to flow at events like this one.

All photos by Mark Lang.

Setup day at NEAF 2008

Vendors were setting up all afternoon here at NEAF. The weather was sunny and warm.


After setting up the Knightware booth, I wandered around to check out vendors and all the wonderful wares. It seems that refractors and observatories are very well represented. A roll-off roof display was being setup that had an operational roof. I could really go for one of those. Maybe after the kids finish college.


I had a great discussion with Normand Fullum of Quebec about mirror making and Dobsonian telescope building. Too bad the 16 inch blank he had was not for sale. With a cool down time of 20 minutes, it's just what I've been looking for in my next scope building project. This is definitely on my to-do list.



I spoke with lots of other interesting people - some known to me only by e-mail or telephone and some not known at all. I had a great talk with Mike Bieler of Astronomics, mostly about Telecaster guitars. He's got one, and I want one. A kindred spirit I think.

It's a wonderful atmosphere here - just like being a kid in a candy store! I'm delighted to be here.

All photos by Mark Lang.

Preparing for NEAF

Getting things ready for NEAF next week has been a lot of work - setting up computers, making handouts, producing discs, etc. I noticed that CloudyNights will be broadcasting interviews with attendees, and it occurred to me that my doing a blog for the event might be fun. Hopefully customers and friends will enjoy attending vicariously. If wireless networking is available in the building, we'll try to upload some reports and photos during the day. If not, we'll post in the evening.

Husband and I aren't looking forward to the long drive, but driving seems more reliable right now than flying. My recent flight to Florida went without a hitch, but then there's all the gear we plan to bring. I'd hate for computers and monitors to end up somewhere other than New York...

If you're at NEAF, please come by the Knightware booth and say hello. I really enjoy putting faces to names known to me only from the ether.