SPURIA // FIELD NOTES

Deep in the Forest: Geolocating a Video by Its Bird Song

Hacktoria GEOINT. Eighteen seconds of bog and spruce with nothing to read in the frame. The picture leads nowhere; the soundtrack is a 2008 xeno-canto recording, and its recordist logged the exact spot on observation.org.

/mediumgeoint/📍 45.58788, -78.37400/#geolocation#hacktoria#audio#bioacoustics#streetview#osint

Title: Deep in the forest 🌲

Description: A very special observation happened at this place. Find out where and at what date the observation was made.

Flag: Hacktoria{Pano_ID_YYYY_MM_DD} — example Hacktoria{CIHM0ogKEICAgID68JS7HA_2011_12_13}

The answer, up front: the place is the Spruce Bog Boardwalk in Algonquin Provincial Park, Ontario, Canada. The “observation” is a singing Veery logged there on 27 May 2008, 05:30. Its recording is the soundtrack of the challenge video. The photosphere that shows the same view as the video is CIHM0ogKEICAgIC4_JrhjwE, taken in October 2016. Hacktoria accepted the flag.

Hacktoria{CIHM0ogKEICAgIC4_JrhjwE_2008_05_27}

The only attachment is location.mp4: 18.3 seconds, 1280×720, pillarboxed, with a mono AAC audio track. The metadata is empty (every date is 0000:00:00) and there is no GPS. The frame has no sign, no road, no building and no text.

The challenge video's first frame: a bog of brown and olive shrubs and sedge in the foreground, a dense wall of dark spruce behind it, a ridge of orange and yellow hardwoods on the right horizon, a strip of open water with lily pads in the bottom-right corner, and a partly cloudy sky

The whole picture. A boreal bog in autumn: spruce, a hardwood ridge turning orange, a bit of open water. The camera is almost static, with a slight handheld drift and wind in the grass. The frame tells you the biome and the season, and nothing more specific than that.

Step 1: The picture leads nowhere

Wrong turn: the Netherlands

Bog, conifers, flat ground: my first guess was the Netherlands. Nothing in the frame confirms that, and nothing in the frame could. No species of tree or shrub here is specific to one country, and there is no infrastructure to check against Plonk It. A reverse image search on a still (Google Lens, “Exact matches”) returns nothing. The frame has never been published anywhere Google indexes.

What the picture does settle is the season: the hardwoods on the ridge are turning, so the video was shot in autumn. That becomes important in a moment.

Step 2: Listen to it

The description says “a very special observation”. In birding, an observation is a logged record: a species, a place, a date. And the soundtrack has a bird on it.

Observation. Running the audio through BirdNET-Analyzer (2.4.0, default model, 3-second segments, no location filter):

ffmpeg -i location.mp4 -vn -ac 1 -ar 48000 audio.wav
birdnet-analyze audio.wav -o out --min_conf 0.1 --rtype csv
Start (s),End (s),Scientific name,Common name,Confidence
0.0,3.0,Catharus fuscescens,Veery,0.9841
6.0,9.0,Regulus satrapa,Golden-crowned Kinglet,0.2512
9.0,12.0,Catharus fuscescens,Veery,0.8924
15.0,18.0,Catharus fuscescens,Veery,0.6330

Spectrogram of the video's audio, 0 to 10.5 kHz over 18 seconds: four boxed strophes between 2 and 5 kHz, each a cascade of descending notes, labelled Veery song; above them, two dashed boxes around faint short upward notes at 6.5 to 8 kHz, labelled Golden-crowned Kinglet

Four Veery strophes, the typical downward spiral between 2 and 5 kHz. Above them, faint and very high, a Golden-crowned Kinglet. Many adult listeners can’t hear it at all at 7–8 kHz, but it shows up clearly in the spectrogram, and it is the detail that settles the match later.

Inference. “Veery special observation” is the pun, and the bird is the lead. The season adds a second point: Veerys sing on their breeding grounds from May to July. A full song over an autumn bog is out of season. So the soundtrack was most likely recorded somewhere else, at another time, and laid over the video. A published recording has a recordist, a date and coordinates, and those are exactly the “where” and “when” the flag asks for.

How firm. BirdNET at 0.98, and the spectrogram is a textbook Veery. Firm. The overlay argument is a hypothesis until the recording turns up. If no recording matches, it fails.

Step 3: The usual databases

The plan: find a public recording whose spectrogram matches this one note for note.

What didn’t work, one line each:

  • Macaulay Library videos: all 164 Veery videos checked by thumbnail. None shows a bog without a bird in it.
  • Macaulay Library audio: the 290 top-rated Veery recordings, none matching.
  • iNaturalist: none of the 137 Veery observations with both a photo and a sound shows this view or comes from the right place and season.

What did work is xeno-canto, which holds 495 Veery recordings:

https://xeno-canto.org/explore?query=Catharus+fuscescens

Comparing 495 files by ear is not realistic, so I compared spectrograms with a script (see Tooling below). Exactly one recording stood out. Most score between 0.3 and 0.45, and XC83778 scores 0.86.

Note: the xeno-canto search box doesn’t find catalogue numbers. Typing XC83778 returns “No recordings found”. Open the record directly at xeno-canto.org/83778.

XC83778 · Veery · Catharus fuscescens · song
Recordist   Jelmer Poelstra
Date        2008-05-27
Latitude    45.919
Longitude   -77.583
Location    Algonquin Provincial Park, Ontario
Country     Canada
Elevation   50 m
Background  Golden-crowned Kinglet Regulus satrapa
Length      25.5 s

The xeno-canto record page for XC83778: a sonogram of the Veery song, and the basic data table with recordist Jelmer Poelstra, date 2008-05-27, latitude 45.919, longitude -77.583, Algonquin Provincial Park, Ontario, elevation 50 m, uploaded 2011-07-19; under sound details the background species is Golden-crowned Kinglet, Regulus satrapa

The record lists a Golden-crowned Kinglet as background species, the same bird BirdNET found faintly above the Veery. It also gives coordinates that turn out to be wrong.

Step 4: The coordinates are wrong, the observation is not

Observation. The xeno-canto coordinates 45.919, -77.583 fall on a forest track near the eastern edge of the park. The listed elevation of 50 m is implausible for Algonquin, which lies at roughly 180 to 600 m. And there is no photosphere within 4 km.

Inference. The coordinates are probably rough, picked from a map when the file was uploaded in 2011. Recordists often log the same observation in a sightings database as well. Jelmer Poelstra is Dutch, and the Dutch database is observation.org (waarneming.nl).

Action. Open observation.org → Veery (/species/76016/) → Observations, filtered to the recording date:

https://observation.org/species/76016/observations/?date_after=2008-05-20&date_before=2008-06-05

Two Algonquin records from 27 May 2008. The first one is observation 49728311:

observation.org record 49728311: Veery, 1 adult male, courtship/singing, 2008-05-27 05:30, observer Jelmer Poelstra, Algonquin Provincial Park; GPS 45.58788, -78.37400; location details Spruce Bog Trail (Pointer at exact spot); the map shows the pin just west of the Spruce Bog Boardwalk Trail label, north of Highway 60; the attached recording below

Same observer, same date, a singing male, a sound attached, and a pin the observer marked as the exact spot. The map puts it at the Spruce Bog Boardwalk, just north of Highway 60.

There it is: the observation, in the challenge’s own word. It has a time, a sound attached, and a precise pin with the remark “Pointer at exact spot”. That pin lies 71.6 km south-west of the xeno-canto coordinates, on the Spruce Bog Boardwalk beside Highway 60.

Step 5: The photosphere

With the Street View layer on, there are several photospheres along the boardwalk around the pin.

False lead: the nearest one

The closest is CIHM0ogKEICAgICkmIWuLQ, 80 m from the pin, on the bridge at the start of the trail. It is a summer pano (July 2016) with bright green sedge and blue sky, and no view from that bridge looks like the video. The challenge asks for “this place”, and the video shows which part of the place is meant. Distance to the pin is not the criterion.

The match

The video was shot in autumn, so the useful candidates are the autumn photospheres: October 2016, October 2018, October 2024. CIHM0ogKEICAgIC4_JrhjwE (Shane Jiang, October 2016, 140 m from the pin, on the boardwalk), looking at a heading of about 316°. The video is more zoomed in than any default pano view, so I registered the two with feature matching (SIFT keypoints on the treeline, 268 RANSAC inliers) and cropped the pano to exactly the video’s framing:

Two images stacked. Above: the challenge frame. Below: the October 2016 photosphere on the Spruce Bog Boardwalk, heading about 316 degrees, cropped and warped to the video's framing. Red boxes at identical positions in both mark the same three features: A, a tall spruce spire above a small red maple; B, the orange hardwood ridge on the right horizon; C, the strip of open water with lily pads in the bottom-right corner

Challenge above, photosphere below. A: the same spruce spire with a red maple at its foot. B: the same orange ridge where the treeline dips. C: the same open water in the right foreground. The boxes sit at identical pixel positions in both halves, and every feature lands inside them. Even the clouds line up, so the video and the photosphere were evidently taken on the same visit in October 2016.

Verification

  1. The audio, over its full length. The video’s 18.3 s align to XC83778 at an offset of 0.53 s. The normalised cross-correlation over the whole track is 0.71. The second-best position anywhere in the same file reaches 0.19, and none of the 784 other Veery recordings tested (494 on xeno-canto, 290 on Macaulay) gets above 0.47. Broken into 2-second windows, every window correlates at 0.67–0.83, including the stretches between strophes. A different Veery singing a similar song can’t do that.

Two spectrograms stacked and time-aligned. Top: the challenge audio. Bottom: XC83778 from 0.53 seconds. Dashed green lines at the start of each strophe and call pass through the same features in both: the four Veery cascades, the flat 4.2 kHz calls between them, the 8 kHz chip at 0.7 s and the run of kinglet notes between 7 and 10 seconds at 7 to 8 kHz

Every strophe, every flat call between strophes and every kinglet note sits at the same time and the same pitch in both. The video track has an extra band of noise between about 1.7 and 4.9 kHz that the original lacks, which is why the full-length score is 0.71 rather than higher.

  1. The place, from an independent source. The video and observation.org know nothing about each other. Spruce Bog came from the observer’s location note, and the photosphere then showed the same spruce spire, ridge and water in the video. Two separate threads end at the same few metres of boardwalk.

  2. Alternatives excluded. The xeno-canto coordinates: forest track, implausible elevation, no photosphere within 4 km, and 71.6 km from the observer’s own pin. The nearest pano (80 m): wrong season and wrong view. The two other autumn photospheres, about 210 m from the pin, don’t fit either: CIHM0ogKEICAgICX--P9PA (October 2024) is an aerial drone panorama, and CIHM0ogKEICAgIDqjcTVtQE (October 2018) stands further along the boardwalk, where the treeline and water are arranged differently.

Moment of certainty: the kinglet. The xeno-canto record lists a Golden-crowned Kinglet as background species. BirdNET heard it in the video. In the aligned spectrograms, its notes at 7–8 kHz fall at the same time in both files. A second Veery recording could resemble this one, but it would not have the same kinglet in the background at the same moments.

Flag

Hacktoria{CIHM0ogKEICAgIC4_JrhjwE_2008_05_27}

Observation: Veery, 1 adult male, singing, 27 May 2008, 05:30, Spruce Bog Trail, Algonquin Provincial Park, 45.58788, -78.37400 (observer’s pin, “exact spot”). Photosphere CIHM0ogKEICAgIC4_JrhjwE, October 2016, about 140 m north-east of the pin. The date in the flag is the observation’s, not the video’s or the pano’s.

Takeaways

  • When the picture is generic, check the audio. A soundtrack carries information the frame doesn’t. A spectrogram takes a second to make and shows sounds you can’t hear.
  • Check whether sound and picture belong together. A Veery singing over autumn leaves is out of season, which means the audio was likely added from somewhere else. An added track has a source, and sources have metadata.
  • Use background sounds to confirm a match. Two songs of the same species can look alike. A second species in the background, at the same moments, is much harder to explain by chance.
  • Treat old upload coordinates with suspicion, and look for a second database. One observation often exists in several places: xeno-canto for the sound, observation.org or eBird for the sighting. The sightings database is usually the more precise one.
  • Choose the pano by what it shows, not by distance. The nearest photosphere was 80 m away and showed something else. The one that matched was 140 m away, in the right season.

Tooling

Spectrogram matching was a short Python script (numpy, scipy, ffmpeg for decoding). Each recording is resampled to 16 kHz and turned into a log-magnitude STFT (512-point window, 128-sample hop), cut to the 1.8–5.5 kHz Veery band, with each frequency row’s median subtracted. Four strophe templates from the video are slid over each recording with FFT-based normalised cross-correlation, and the best score per file is kept. The core:

def ncc(S, t):                      # S: recording spectrogram, t: template
    F, w = t.shape; n = F * w
    tn = (t - t.mean()) / t.std()
    num = fftconvolve(S, tn[::-1, ::-1], mode="valid")[0]
    ones = np.ones((F, w), np.float32)
    s1 = fftconvolve(S, ones, mode="valid")[0]
    s2 = fftconvolve(S * S, ones, mode="valid")[0]
    sd = np.sqrt(np.maximum(s2 / n - (s1 / n) ** 2, 1e-9))
    return (num / n / sd).max()

As a sanity check, an 18-second excerpt of a random xeno-canto file, re-encoded to AAC, scores 0.995 against its source. Xeno-canto files download directly from https://xeno-canto.org/<id>/download. Request them slowly, because the server returns 429 Too Many Requests under parallel load.

Sources