Merge branch 'master' of http://github.com/twpayne/igc2kmz

HACKING.md

@@ -0,0 +1,65 @@
+Photo placement
+---------------
+
+File: `igc2kmz/__init__.py`
+Function: `Flight.make_photos_folder`
+
+igc2kmz uses the EXIF information embedded in the photo to place at the location it was taken. If the EXIF information includes a GPS position then this is used, otherwise the tracklog is examined to find the pilot's location when the photo was taken.
+
+This of course assumes that the user has set the time and date on his camera correctly, which unfortunately many people do not do. Also, if they do set it, they tend to set it to local time at home, which can be quite different from local time at the time and place of the flight and is not UTC. Unfortunately the EXIF information does not record any timezone, so we don't have complete information and the photos can end up placed incorrectly.
+
+
+Thermal and glide analysis
+--------------------------
+
+File: `igc2kmz/track.py`
+Function: `Track.analyse`
+
+To identify thermals and glides a simple but effective heuristic is used. The average climb rate over 20 seconds is compared with the pilot's progress. Progress is defined as the distance travelled along the track divided by the change in position over the same period. For example, when flying in a straight line the progress is close to one, but if the pilot circles then progress drops to close to zero: the pilot travels a long distance along the track without covering much distance over the ground. Experimental investigation suggests that progress values over 0.9 correspond to gliding behaviour, and values less than this correspond to thermalling or emergency descent behaviour, even in strong winds.
+
+Therefore we can classify the track using the following:
+
+* progress > 0.9 ⇒ gliding
+* progress < 0.9 and climb rate > 0 ⇒ thermalling
+* progress < 0.9 and climb rate < 0 ⇒ emergency descent
+
+Further refinements include ensuring that the identified features are of interest to the pilot, that is that glides are long enough, a significant amount of height is gained in a thermal, and so on.
+
+
+Average, maximum and peak climb rates and thermal efficiency
+------------------------------------------------------------
+
+File: `igc2kmz/__init__.py`
+Function: `Flight.make_analysis_folder`
+
+The average climb rate is the calculated for the entire thermal, that is the total height gained divided by the time taken. Maximum climb rate is the maximum climb rate on a 20 second average. Peak climb rate is the highest climb rate observed between sequential points in the track log.
+
+The thermal efficiency is the average climb rate divided by the maximum climb rate. A thermal efficiency of 100% corresponds to flying straight into the strongest core and staying in it until exiting the thermal. Lower values correspond to spending less time in the core or losing the thermal completely at times. This simple model assumes that the maximum climb rate is achievable from the start of the thermal to its end which is rarely the case, usually the thermal strength varies with height depending on the airmass.
+
+In practice, thermal efficiencies over 80% are rare, 70% or higher is very good, and anything below 50% indicates broken thermals and/or poor thermalling technique.
+
+
+Salient altitude analysis
+-------------------------
+
+File: `igc2kmz/util.py`
+Function: `salient`
+
+The pilot is often interested in his maximum or minimum altitude at various points. However, highlighting every local minima and maxima leads to an overwhelming number of points. The salient algorithm uses a divide and conquer technique to find all pairs of consecutive maxima and minima where the difference between them is greater than a certain threshold. Broadly speaking it proceeds as follows:
+
+* for a given sequence x[i]..x[j], if the overall trend is upwards (i.e. x[i] < x[j]) then find the largest drop in the sequence, that is find (m, n) that maximises x[m] - x[n] subject to m < n
+* if the overall trend is downwards (i.e. x[i] > x[j]) then the largest climb in the sequence, i.e find (m, n) that minimises x[m] - x[n] subject to m < n
+* if the overall trend is flat (i.e. x[i] == x[j]) then compute candidate values of (m, n) using both the above and chose the value of (m, n) that maximises | x[m] - x[n] | (i.e. find both the largest drop and the largest climb and choose which ever is bigger)
+* if the magnitude of this change is less than our threshold then we are done
+* otherwise add m and n to the set of salient points and recurse with the sub-sequences i..m, m..n, and n..j
+
+In the worst case the algorithm is O(N^2), but in the normal case is O(N log N). An obvious speed-up is to pre-filter the sequence to remove all monotonic sub-sequences but this has not proved necessary with the length of sequences used in the program.
+
+
+Spherical geometric functions
+-----------------------------
+
+File: `igc2kmz/coord.py`
+Function: `Coord.initial_bearing_to`, `Coord.distance_to`, `Coord.halfway_to`, `Coord.interpolate`, `Coord.coord_at`
+
+All these geometric formulae are taken from this excellent page of [spherical geometry formulae](http://www.movable-type.co.uk/scripts/latlong.html). Note that all distance calculations assume that the Earth is a perfect sphere with the FAI radius (r=6371km).

README.md

@@ -0,0 +1,59 @@
+igc2kmz IGC to Google Earth converter
+=====================================
+
+igc2kmz converts paraglider and hang glider track logs into Google Earth KML format with lots of features, notably:
+
+* track colored by altitude, climb rate and ground speed
+* shadow feature makes it easier to judge the track's altitude by eye
+* animation of the flight
+* photos automatically placed where they were taken and with an optional comment
+* XC optimisation output
+* altitude graph and high and low points labelled
+* thermal and glide analysis
+* time marks
+
+It's used by the following XC league servers:
+
+* [Leonardo](http://www.paraglidingforum.com/leonardo)
+* [UK National XC League](http://www.uknxcl.org.uk/)
+
+Just upload your flight to one of these and you can download your flight in Google Earth format without having to install any extra software on your computer.
+
+It is designed to run on XC league servers, and as such is not designed to be directly used by pilots.
+
+
+Requirements
+------------
+
+* [Python](http://www.python.org/) version 2.5 or 2.6, not version 3.0
+
+
+Get the code
+------------
+
+Download either the [zip archive](http://github.com/twpayne/igc2kmz/zipball/master) or the [tar.gz archive](http://github.com/twpayne/igc2kmz/tarball/master).
+
+Unpack this archive somewhere.
+
+If you want to track development then you can checkout the source code with [git](http://git.or.cz/) instead of downloading an archive:
+
+	git clone git://github.com/twpayne/igc2kmz.git
+
+
+Run it
+------
+
+Change to the directory where you unpacked the archive and run:
+
+	bin/igc2kmz.py -i <input-filename>.igc -o <output-filename>.kmz
+
+
+Customise it
+------------
+
+You can set various parameters via the command line, including the time zone offset in hours relative to UTC. For example, use `-z 2` for Central European Time during the summer. For individual flights you can override the pilot name and glider type (otherwise they are taken from the IGC file), set the line color and width, add optimized XC information and photos with comments. Run `bin/igc2kmz.py --help` for a full list of options. For example use, look at the Makefile.
+
+Rebuild the examples
+--------------------
+
+You can rebuild the example files with the command `make examples`. This will build the `olc2002` flight optimizer, optimize a number of flights, and create the KMZ files in the examples/ subdirectory. Be warned that the flight optimization step can take a long time (30 minutes on a 2.4GHz Core 2 Duo).

bin/leonardo2kmz.py

@@ -41,6 +41,7 @@ DEFAULT_DIRECTORY = '/var/www/html'
 DEFAULT_TABLE_PREFIX = 'leonardo'
 DEFAULT_IGC_PATH = 'data/flights/tracks/%YEAR%/%PILOTID%'
 DEFAULT_PHOTOS_PATH = 'data/flights/photos/%YEAR%/%PILOTID%'
+DEFAULT_PHOTOS_URL = '/modules/leonardo/data/flights/photos/%YEAR%/%PILOTID%'
 
 LEAGUE = (None, 'Online Contest', 'World XC Online Contest')
 ROUTE_NAME = (
@@ -138,6 +139,8 @@ def main(argv):
                       help='set IGC path')
     parser.add_option('-P', '--photos-path', metavar='STRING',
                       help='set photos path')
+    parser.add_option('-U', '--photos-url', metavar='STRING',
+                      help='set photos URL')
     parser.set_defaults(output='igc2kmz.kmz')
     parser.set_defaults(name=DEFAULT_NAME)
     parser.set_defaults(icon=DEFAULT_ICON)
@@ -147,6 +150,7 @@ def main(argv):
     parser.set_defaults(table_prefix=DEFAULT_TABLE_PREFIX)
     parser.set_defaults(igc_path=DEFAULT_IGC_PATH)
     parser.set_defaults(photos_path=DEFAULT_PHOTOS_PATH)
+    parser.set_defaults(photos_url=DEFAULT_PHOTOS_URL)
     parser.set_defaults(igc_suffix='.saned.full.igc')
     options, args = parser.parse_args(argv)
     #
@@ -182,7 +186,8 @@ def main(argv):
                 'PILOTID': str(pilot_id),
                 'YEAR': str(flight_row.DATE.year),
                 }
-        igc_path = os.path.join(substitute(options.igc_path, substitutions),
+        igc_path = os.path.join(options.directory,
+                                substitute(options.igc_path, substitutions),
                                 flight_row.filename + options.igc_suffix)
         track = IGC(open(igc_path), date=flight_row.DATE).track()
         flight = Flight(track)
@@ -235,10 +240,12 @@ def main(argv):
             select = photos_table.select(photos_table.c.flightID
                                          == flight_row.ID)
             for photo_row in select.execute().fetchall():
-                photo_url = options.url + PHOTO_URL % photo_row
-                photo_path = os.path.join(substitute(options.photos_path,
-                                                     substitutions),
-                                          photo_row.path, photo_row.name)
+                photo_url = options.url \
+                        + substitute(options.photos_url, substitutions) \
+                        + '/' + photo_row.name
+                photo_path = os.path.join(options.directory,
+                                          substitute(options.photos_path, substitutions),
+                                          photo_row.name)
                 photo = Photo(photo_url, path=photo_path)
                 if photo_row.description:
                     photo.description = photo_row.description

igc2kmz/gpx.py

@@ -25,6 +25,7 @@ except ImportError:
 
 from coord import Coord
 from track import Track
+from waypoint import Waypoint
 
 
 GPX_DATETIME_FORMAT = '%Y-%m-%dT%H:%M:%SZ'
@@ -60,6 +61,7 @@ class GPX(object):
         element = parse(file)
         namespace = re.match('\{(.*)\}', element.getroot().tag).group(1)
         ele_tag_name = '{%s}ele' % namespace
+        name_tag_name = '{%s}name' % namespace
         time_tag_name = '{%s}time' % namespace
         self.coords = []
         for trkpt in element.findall('/{%s}trk/{%s}trkseg/{%s}trkpt'
@@ -74,6 +76,15 @@ class GPX(object):
             dt = datetime.strptime(time.text, GPX_DATETIME_FORMAT)
             coord = Coord(lat, lon, ele, dt)
             self.coords.append(coord)
+        self.waypoints = []
+        for wpt in element.findall('/{%s}wpt' % namespace):
+            name = wpt.find(name_tag_name).text
+            lat = math.pi * float(wpt.get('lat')) / 180.0
+            lon = math.pi * float(wpt.get('lon')) / 180.0
+            ele_tag = wpt.find(ele_tag_name)
+            ele = 0 if ele_tag is None else float(ele_tag.text)
+            waypoint = Waypoint(name, lat, lon, ele)
+            self.waypoints.append(waypoint)
 
     def track(self):
         return Track(self.coords, filename=self.filename)

igc2kmz/igc.py

@@ -28,6 +28,7 @@ A_RECORD_RE = re.compile(r'A(.*)\Z')
 B_RECORD_RE = re.compile(r'B(\d{2})(\d{2})(\d{2})(\d{2})(\d{5})([NS])'
                          r'(\d{3})(\d{5})([EW])([AV])(\d{5})(\d{5}).*\Z')
 C_RECORD_RE = re.compile(r'C(\d{2})(\d{5})([NS])(\d{3})(\d{5})([EW])(.*)\Z')
+E_RECORD_RE = re.compile(r'E(\d{2})(\d{2})(\d{2})(\w{3})(.*)\Z')
 G_RECORD_RE = re.compile(r'G(.*)\Z')
 HFDTE_RECORD_RE = re.compile(r'H(F)(DTE)(\d\d)(\d\d)(\d\d)\Z')
 HFFXA_RECORD_RE = re.compile(r'H(F)(FXA)(\d+)\Z')
@@ -62,6 +63,11 @@ class Record(object):
 
     __metaclass__ = Metaclass
 
+    def __repr__(self):
+        return '%s(%s)' % (self.__class__.__name__,
+                           ', '.join('%s=%s' % (key, repr(value))
+                                     for key, value in self.__dict__.items()))
+
 
 class ARecord(Record):
 
@@ -78,8 +84,6 @@ class ARecord(Record):
 
 class BRecord(Record):
 
-    __slots__ = ('dt', 'lat', 'lon', 'validity', 'alt', 'ele')
-
     @classmethod
     def parse(cls, line, igc):
         result = cls()
@@ -87,9 +91,18 @@ class BRecord(Record):
         if not m:
             raise SyntaxError, line
         for key, value in igc.i.items():
-            setattr(result, key, int(line[value]))
+            try:
+                setattr(result, key, int(line[value]))
+            except ValueError:
+                setattr(result, key, None)
         time = datetime.time(*map(int, m.group(1, 2, 3)))
+        if 'tds' in igc.i:
+            time = time.replace(microsecond=int(line[igc.i['tds']]) * 100000)
         result.dt = datetime.datetime.combine(igc.hfdterecord.date, time)
+        if igc.b and result.dt < igc.b[-1].dt:
+            igc.hfdterecord.date = datetime.date.fromordinal(
+                    igc.hfdterecord.date.toordinal() + 1)
+            result.dt = datetime.datetime.combine(igc.hfdterecord.date, time)
         result.lat = int(m.group(4)) + int(m.group(5)) / 60000.0
         if 'lad' in igc.i:
             result.lat += int(line[igc.i['lad']]) / 6000000.0
@@ -126,6 +139,18 @@ class CRecord(Record):
         return result
 
 
+class ERecord(Record):
+
+    @classmethod
+    def parse(cls, line, igc):
+        result = cls()
+        m = E_RECORD_RE.match(line)
+        if not m:
+            raise SyntaxError, line
+        result.value = m.group(4)
+        return result
+
+
 class GRecord(Record):
 
     @classmethod
@@ -177,8 +202,8 @@ class IRecord(Record):
             m = I_RECORD_RE.match(line, 3 + 7 * i, 10 + 7 * i)
             if not m:
                 raise SyntaxError, line
-            igc.i[m.group(3).lower()] = slice(int(m.group(1)),
-                                              int(m.group(2)) + 1)
+            igc.i[m.group(3).lower()] = slice(int(m.group(1)) - 1,
+                                              int(m.group(2)))
         return result
 
 
@@ -240,3 +265,8 @@ class IGC(object):
             if any(getattr(b, k) for b in self.b):
                 kwargs[k] = [getattr(b, k) for b in self.b]
         return track.Track(coords, **kwargs)
+
+
+if __name__ == '__main__':
+    import sys
+    print repr(IGC(sys.stdin).__dict__)

igc2kmz/waypoint.py

@@ -0,0 +1,32 @@
+#   igc2kmz waypoint functions
+#   Copyright (C) 2010  Tom Payne
+#
+#   This program is free software: you can redistribute it and/or modify
+#   it under the terms of the GNU General Public License as published by
+#   the Free Software Foundation, either version 3 of the License, or
+#   (at your option) any later version.
+#
+#   This program is distributed in the hope that it will be useful,
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#   GNU General Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License
+#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from math import pi
+
+from coord import Coord
+
+
+class Waypoint(Coord):
+
+    def __init__(self, name, lat, lon, ele, description=None):
+        Coord.__init__(self, lat, lon, ele)
+        self.name = name
+        self.description = description
+
+    @classmethod
+    def deg(cls, name, lat, lon, ele, description=None):
+        return cls(name, pi * lat / 180.0, pi * lon / 180.0, ele, description)