I’ve pushed some changes to the pypeline repository, adding basic stacking functionality. There isn’t any registration, it only takes the median of each channel (R, G, B) for each pixel. It’s currently way slow, but I suspect there is substantial room for improvement there. Wrangling NEF files has proven more difficult than I anticipated, so currently the state of the art in pypeline is JPGs.
My camera is rated down to 32°F, and nightly lows have been around 0, so I’m scared to take it out into the elements. On the plus side, the stacking works with regular images too! Any particular pixel just needs to have the “right” value for at least half the shots.
And the stacked result:
There is a little ghosting, but quite good considering, I’d say. I am not sure how to get rid of that totally. More pictures should quash the error, but at 5 I would have thought it would wipe out any traces of the marker. Also, a better algorithm should be able to push down the > 50% requirement to only a plurality. Maybe with some sort of clustering of values? I’m also taking the mean of each channel independently, maybe a better way would be to use luminance. In any case, baby steps!
Inspired by this post at DataGenetics, I implemented a quick-and-dirty script in python to test it out. The first takes an input image and iterates over it pixel by pixel, splitting it into two output images. Ideally, the outputs are randomly assigned, so it is impossible to recover the original without both outputs. The outputs can be combined to recover the original. Here’s an example of it in action:
Intermediate images (hopefully look like static):
Not perfect, but it is definitely recognizable. The idea can apparently be extended to 9×9 (and 16×16, and 25×25… I presume) images, for a wider-shared secret. In any case, this scheme should make it possible for any number of people to share a secret, but none of them individually can recover it. I uploaded the code here on github.
I’ve been a little busy lately, but should be able to pick things up again with the new PC I built. Made a time-lapse of assembling it too:
I’ve just enabled SSL on the site. Technically it’s TLS, but the name SSL seems to be sticking. Now you can access it at https://www.foxrow.com (note https). The not-encrypted version should still be available at http://www.foxrow.com. I’m not sure with Heroku hosting the apps if I can get it set up for ergs and weather, still looking into that.
Apparently PIL, and therefore Pillow, do not support Nikon RAW (.NEF) files. From what I can tell, my camera shoots RAWs in a 14-bit grayscale format. With the help of nefarious, I’ve found a way to get the image data out of my RAW files. Images coming soon!
As part of my ongoing effort to consolidate my various projects onto foxrow.com, I have moved the rowing weather app to weather.foxrow.com. Eventually I hope to get some interactive form of pypeline online as well. I am not sure yet what form that might take. Because RAWs are so big, uploading enough to get a reasonable outcome may be prohibitively bandwith-intensive. Also processing power on the server side may prove too big a bottleneck, but I won’t know until I try it.
I’m using Linux Mint 15 (“Olivia”) for my pypeline development – and ran into a little difficulty right off the bat getting pillow installed. Virtualenv and Virtualenvwrapper are pretty much a must-have if you’re developing multiple projects, and
pip install pillow was failing with messages like
Python.h: No such file or directory despite following the installation instructions to a T. Turns out since I’m using Python 3,
sudo apt-get install python-dev doesn’t cut it. What you want is
sudo apt-get install python3-dev. Run that once and you should be good to go in your virtualenv.
Since I plan on taking quite a few night sky pictures, I’d like to automate the process as much as possible. The project is hosted on github here. Right now it’s essentially empty, but eventually features I’d like it to have:
- Image calibration – support for flat frames, bias frames, dark frames
- Image registration
- Image stacking
- An optional GUI
Why roll your own pipeline, when other programs exist?
I’d like to learn more about the process and get some experience with astrophotography.
The majority of stacking/alignment applications I’ve found are windows-only.
Why open source?
The majority of stacking/alignment applications I’ve found are closed-source, even if they’re free as in beer. I like free as in everything.
What formats/algorithms do you support?
Right now, none. Since my camera is a Nikon D5200, the first thing I’ll implement is Nikon RAW (NEF) files. Various stacking/averaging algorithms will have to come later.
What’s your release schedule?
When I get around to it. There’s no hard timetable on anything, but I’d like to get a good chunk implemented sooner rather than later. Got some pictures to process!
Python 3? Really?
Yes, really. Python 3 is the future of the language, and Django and PIL were the two major things holding me back. Well, now Django is py3k compatible, and now we have pillow, the friendly PIL fork. Lucky!
That’s all for now, follow the project on github or fork it and contribute!
With one set of exposures under my belt, I adjusted my camera settings a little and tried my hand again. As luck would have it, my window framed the big dipper quite nicely, so I set up in front of that and let it rip.
- 51 light frames
- 10 second exposures
- ISO 1000
- 18mm focal length
- 31 dark frames
- 31 bias frames – same everything except exposure time: 1/4000th second
Stacked in deep sky stacker and processed in Picasa. I really need to get some better software. This was closer to the horizon than the previous shot, so there was heavy light pollution near the bottom of the frame. After abusing the stacked result pretty severely, I ended up with a recognizable big dipper and black sky. I ended up cropping a fair amount, so the effective focal length is quite a bit longer than 18mm. I think it turned out fairly good, considering:
Well, sort of. This is my first attempt at any sort of serious astrophotography. I wasn’t aiming for anything in particular, I just wanted to get a feel for my camera. The conditions were subpar to say the least: it was a mildly cloudy night, and I’m near a metropolitan downtown, so the light pollution was terrible. Collection details:
- Nikon D5200 w/18-55mm Nikkor lens – RAW (NEF) format
- Focal length: 55mm
- Exposure time: 10secs
- ISO 100
- 31 light frames, 11 dark frames
- Stacked with Deep Sky Stacker
Considering all that, my results weren’t too terrible:
Looks pretty, but that’s about it. Technical note: I was having trouble getting my RAWs to align in DSS. I suspect my ISO was way too low for it to detect stars, but after converting to jpg it seemed to do reasonably well.