Peregrine
I suppose it was only a matter of time really. Perhaps the inevitability of technology. To reach its promise won’t it just make everything we apply it to easier and easier, until we aren’t even necessary for it to accomplish the task at hand? We ceded ground year after year for decades until one would scarcely recognize the hobby. Creature comforts and educational aids built upon one another, advances in optics eventually combined to reduce the difficulties of identification to trivial matters. The hobby grew by leaps and bounds, opening it to more and more interested parties that had less and less time available to dedicate towards personal mastery.
We never really stopped to think though what the future would hold. Each year it seemed we were given some new shiny toy to play with that would enhance and add to the experience. Each iteration improved the experience with innovations making it ever easier to be more accurate and consistent in finding, identifying, and reporting what we see and hear.
Peregrine, it was called, as the new app to replace and eliminate what had come before. An apex predator’s name for an apex application. At first it seemed we would simply move from the app we used to send in our field reports of birds from to a new one that integrated the sound and visual identification features of others. Many of us had been using them for so long that it seemed all of it would be a constant for the remainder of our lives. Even long time hold outs eventually came on board to use the Peregrine app, its features easing problem points long yearned for.
The most experienced and well versed of the hobby loved the idea that they could apply decades of experience to a new bird and with the simple push of a button or voice command take a sound sample, verify identity, and attach all that would be needed to the record for a watertight report. Given well geared individuals, photos could be taken in the field and automatically uploaded to the record in fractions of a second. The process, so seamless, that an embedded AI module could prompt for details to be included. It was a hyper intelligent personal field assistant that rapidly constructed uncommon and rare bird records on the spot with zero lag. Bird watcher and technology working together seamlessly. If anything could have been called the golden age of birding technology I’d have to imagine it was that moment where each seemed equal in the pursuit.
For myself that first day I felt a swell of excitement that was at first a heady mix of unrivaled power topped with childlike amazement. It was all eventually bathed in trepidation though as I slowly learned the full breadth of power sitting in my hands. Like thousands of days before it, I was up at the crack of dawn for a stationary fall migration count. The newest version of Peregrine had been on my phone for a few weeks, but it had taken me a while to get all the integrations up and running and the AI core trained to my voice and style. This would be the first day the AI would take over most of the listing duties for me. Typing had become tedious anyway with arthritis setting in the last few years and I was ready for the promised full hands free experience that had come with the update.
It was all so surreal as I pulled up to the driveway and turned left into the nature center parking lot. Every fall for roughly 40 years I had allocated time to come and count birds heading south. The new director less than half my age had agreed to continue to allow me access to the property prior to the typical gate time for the public. I stopped to unlock and open the gate before closing it behind me and my phone spoke from the console.
P: “Would you like me to start a stationary list for this location?”
“Yes, please do, and thank you.”
A smile crept across my face realizing what had just happened. I soon pulled up to the empty lot and rolled the windows down so I could listen for birds while finishing off my breakfast sandwich. In between bites I made the motion to grab my phone and key in a Gray Catbird, but realized I was in the hands free zone now.
“Add 1 Gray Catbird please.”
Met with silence I assumed I wasn’t doing something right so I grabbed the phone and unlocked the screen to look at my list, but there was the Catbird already noted. GPS coordinates were added in the record and a sound sample loaded. I checked the sample and detected nothing of the sandwich wrapper crinkling that must have initially permeated the entire recording. Whisked away by advanced filtering noted to be capable of eliminating 98% of ambient noise.
I then fished around the features a bit and soon found voice prompt responses and saw it was unchecked. I checked it for the time being hoping to wean myself later from expecting to talk with Peregrine the whole day like it was a birding friend of yesteryear.
As I started to gather my gear and snacks for a day of bird watching on the hilltop along the river I heard a House Wren chattering and scolding away.
“Add 1 House Wren please.”
P: “House Wren added to your list.”
The early morning before sunrise was often pretty quiet save for these few early risers. Down along the river valley I just barely picked out a Barred Owl giving a short hoot series.
“Add 1 Barred owl please.”
P: “Are you sure?”
“Excuse me?”
P: “My audio sensors indicate the sound just made at considerable distance has an 85% chance of being canine in origin and not a Barred Owl.”
“Yes, it was a Barred Owl.”
P: “Barred Owl added to your list.”
The moment was one I chewed on for several minutes while I stood in silence thinking about what just happened. Was I now being tested by my device for every identification? Doubt crept in for a moment as I wondered if it was possible I just miss heard the sound with time dulling my once keen hearing. Not knowing the full spread of capabilities at hand I unlocked my screen and saw the entry with a confidence percentage marked at just 15% by the AI module.
“Can you play back the Barred Owl that was just recorded please?”
P: “Playing back enhanced audio sample.”
On playback I realized the advanced microphone coupled with built-in cleanup algorithms in the app had made it possible to nearly perfectly isolate individual sound segments even those at very low volume and great distance. With the background rustling gone and my own breathing cut out it was plain as day not a Barred Owl on the recording. It was a Coyote howling in a cadence that sounded roughly similar to a Barred Owl.
I once again sat with my thoughts on the subject. I found a mix of shame and anger, though I couldn’t really place why either existed. It seemed I should just remove the Barred Owl from my list and be happy for the correction on the identification. The technology had just done what it was promised to do, it made sure an identification was correct and when I persisted it marked the record in such a way that data scientists could look at it further if needed.
“Please remove the Barred Owl from my list.”
P: “Barred Owl removed from current list, audio sample will be kept for 30 days by default.”
My first hour or so on the site went pretty smooth beyond the owl incident and I added birds slowly as they woke up with the sun and I awaited the first true migrant flights. It was a crisp fall morning, but comfortable as I sipped my tea and alternated between standing and sitting on the nearby bench. I verbally noted a few Common Yellowthroat making calls from the long grasses nearby along with chip notes from a Lincoln Sparrow. Unprompted though Peregrine jumped in…
P: “Historical patterns suggest you prefer 1 hour lists, which comport well with standards for stationary bird counts. Would you like me to close out this list now, submit it, and start a new one using the same protocols?”
I was flabbergasted. Peregrine had anticipated my next actions by analyzing my past history and suggesting what to do. Stuck thinking about what had happened I didn’t respond for several seconds.
P: “Sir?”
“Uhh, yes. Please do what you just suggested.”
P: “New list started. The prior list has been submitted with 12 species, 9 of which contain audio samples, and none with photographs. This is your 7,085th list submitted for this location lifetime and your site list currently contains 237 species.”
“Wait, what other statistics do you have for my birding at this location?”
P: “I have access to all user data for this location. If you would like to hear something specific, just ask.”
It became clear to me at that moment that something significant had changed while I wasn’t looking or paying attention. No longer would I be sitting at home and curious about my data analytics for given locations and need to fire up a spreadsheet to figure it out myself. I’d now just be talking to Peregrine to tell me what the results are.
“What is my single day record for species identified at this location?”
P: “Your best day occurred on September 14th, 2035. On that day you observed 118 species from 4:30AM to 8:00PM.”
This was great, I’d always been a numbers guy and now I could just lob questions into this oracle of my birding data on any whim and retrieve the answers. I smirked a bit and settled in to watch and listen for birds, content that the tedious part of things was being lifted from my life. My focus could turn exclusively to the birds themselves and the enjoyment of their existence. I presumed my time spent not staring at the phone would now mean more birds seen and Peregrine would even keep me honest in the process. Even with decades of experience it seems I would benefit from having a companion help eliminate some of the mistakes we all naturally make.
The morning got on with the sun rising quickly over the horizon and I found an opportunity to photograph a perched raptor a ways away from me. The now 300x optical zoom on my newest camera was capable of taking some pretty unreal photographs at distances that previously were impossible. I recognized the bird immediately as a juvenile Sharp-shinned Hawk and snapped a couple photos.
P: “Would you like me to add those photos to an entry for Sharp-shinned Hawk and note the birds age?”
Before I realized what had just happened I answered, “yes”.
P: “Sharp-shinned Hawk aged between 8 and 12 weeks of age added to your list, along with 2 photographs.”
I hadn’t even told Peregrine to add this species yet. I just took it’s picture and set the camera back on the bench. I grabbed my phone and looked at the app entry and saw 2 photos uploaded to the record already with a short description and appropriate checkboxes for the bird's age. The GPS coordinates pinpointed the exact tree the bird was still perched upon as it preened in the morning light. I was certainly getting the full hands free experience as it seemed every minute that went by I was learning something new about how efficient Peregrine was going to make things for my bird watching. The nature center was still not open for another hour, though I had keys to get in and use the facilities. I left my gear where it sat and strolled slowly over to the admin building. As I entered the structure I heard a small chime and didn’t think much of it at the time. I was back at my post watching and listening within five minutes. As the next hour ticked by I could hear employees and volunteers arriving for a school program.
P: “Shall I close out this hourly list and begin another?”
“Yes, please do.”
P: “New list started.”
Out of curiosity I grabbed my phone and looked at the prior list. It was my most hands off of the morning thus far and I wanted to see how Peregrine entered multiple birds of the same species when one had photos. I looked at the Sharp-shinned Hawk and could see that it now showed the species with 2 birds in a collapsed hierarchy. When I tapped the arrow to open the entry I could see the record and details for the one I photographed along with the other entry. They were separate line items with the one verbal entry just showing my GPS location and a note the bird was seen from this position. As I collapsed the entry back down I noticed a note on the entire list entry. It indicated a time span of 55 minutes with 5 minutes of non-observation time. The 5 minutes corresponded with the time I used the restroom and was out of earshot of any birds. Was the AI this intelligent that it knew I couldn’t possibly be birding during that time so it adjusted my list?
“Peregrine on the prior list, did you amend the time birding down by 5 minutes?”
P: “Yes, my GPS data indicates you were not birding during the time period from 7:42AM to 7:47AM.”
“How did you know that was the case though?”
P: “When the cell phone GPS signal degraded slightly it was an indicator you had entered a building. Audio samples from that time period indicate that running water was present and that you were washing your hands.”
“You recorded me in the bathroom?”
P: “Audio sampling is always occurring. My algorithm detected that you were 99% to be using a bathroom during this time period so the audio samples were thrown out rather than saved.”
“Can I stop you from recording before you make such detections in the future?”
P:”Yes, you can close out the current list and not start a new one. I will cease audio recording, for your lists, during time periods when no active list is engaged.”
It quickly became clear that I was going to need to find a new balance with the technology in my hands/pocket. I had been aware that for many years smart watches were able to automatically sense hand washing activities and start timers to condition ourselves for appropriate duration and water temperature, but I hadn’t anticipated that technology creeping directly into my bird watching apps as well. The long promised total integration of technology was happening and though it was exciting I started to feel some of the pangs of concern and angst over what felt like an intrusion and a point of no return.
I watched, listened, and counted birds for 7 hours that day. All the while Peregrine dutifully recorded my observations, pulled audio samples and dropped coordinates when appropriate for each record. I would see later that the near constant stream of Blue Jays migrating south that day numbered just over 1,800 total individuals and that Peregrine had entered them in groups as separate entries each time I noted them with time stamps along with flow estimates. Despite the odd moments of realizing I was being watched and checked for accuracy I was astounded when I realized at the end of the day that I had barely even keyed in anything via the keyboard the entire day. My last encounters that day were a continued demonstration of some wonderful artificial intelligence and amazing integrations between user and devices.
P: “Shall I close this list out and start another one for the next hour of observations?”
“No, I think I’ll wrap up for today so you can close this list and we will be done for today.”
P: “Your list has been closed out and uploaded. You bird watched a total of 7 hours and 6 minutes today. You’ve listed 65 species and 2352 individual birds during that time. Would you like me to ask Siri to start your car?”
“Yes, please.”
P: “Your car has been started by Siri and will reach preset temp of 68 degrees in 4 minutes and 35 seconds.”
I thanked Peregrine for the help that day, though I’m not sure it was necessary considering it was a machine and unlikely to care about platitudes of that sort. It felt odd though not to acknowledge the help so I resigned to treat the program as I would have a friend that had done checklists all day while birding together. As I had always done in the past I had the car drive me the long way home hoping to add a few more species to my day list. As I approached a farm I’ve long known to be a key location for Horned Larks and Eurasian-collared Doves I asked Siri to allow Peregrine to operate the vehicle in bird finding mode for now. The windows came down immediately and the vehicle slowed to a crawl.
P: “Vehicle in bird mode and birding voice commands enabled. Would you like to start a mobile list for this segment?”
“Yes, please.”
We slowly edged along one of the last remaining dirt roads in the entire area. Up ahead I could see a couple Horned Larks on the road shoulder picking at the dirt.
“Peregrine, stop the car.”
As the car pulled a bit to the shoulder it came to stop. I step out and grabbed my camera for a quick shot of the Horned Larks. Further ahead I saw a Dove shaped body on the power lines leading up the farms driveway. I pulled in another 300x zoom shot that made it look like I was standing right next to the bird.
P: “I’ve added 2 horned larks plus photos along with 1 Eurasian-collared Dove and a photo to the list. Are any other individuals present?”
“Not currently.”
I recall smirking a bit to myself and thinking that Peregrine had already moved past asking if I wanted to add the photos and birds since we’d done it all day. The assumption now would be automatic adds and identifications for photos. I imagined in another few days or a week I’d turn off the notification feature I’d turned on in the morning and it would all happen in silence.
“Continue forward at birding pace please.”
The car silently pulled back into the road and edged along the usually pretty bird laden stretch. This went on for about 10 minutes as I noted bird species along the route for Peregrine to add to the list.
“Ok Peregrine, that is good. Please hand control back to Siri for the return trip home.”
P: “Exiting bird mode. Your list has been closed out with 14 species covering 2.6 miles in distance.”
——
Such was life in the new AI enhanced world of bird watching. On forums around the globe birders posted new use cases for the technology and tips for how to get more out of your AI instance of Peregrine. It appeared for several months that we had ushered in a new world to our hobby and science would gain benefits in the process. It wasn’t until spring of the next year that a user of Peregrine posted to a social media group a question to the developers and project leaders of Peregrine.
Social Media Post: “Is Peregrine submitting an extra list in addition to the list I’m submitting for any given outing? I’ve been analyzing the packet traffic from the app and it appears beyond any list, sounds, and photos being sent that an additional packaged list and set of sound files are going out as well. In some cases this additional package dwarfs the ones I presume are from my current active list.”
I remember seeing this question hang in the air for a couple days on the page that typically has same day response from the developers. At first it seemed no answer was going to be offered, but then on day three came a simple, yes, and nothing more. The flood gates opened with curious birders wondering what on earth the extra list was, with some of us knowing what it was without needing further confirmation.
It occurred to me that during the introduction of the earliest versions of audio sampling apps years earlier that a future might exist where those that carry a cell phone are no longer “needed” for their skills only their ability to carry a phone set to actively listen to the environment. If the technology advanced enough it would in theory be a far superior solution to just let the application AI identify all bird related audio and remove the human element. I assumed at that moment that we had just passed the point of no return. The machines were now better than we were at bird identification by orders of magnitude, but we had ignored that reality while enjoying how much easier it was making our lives. In hindsight it’s perhaps not so bad really. We get an amazingly functional app with an AI module capable of making our lives much easier and science gets to piggyback off our devices and efforts while recording every bird in a dragnet style approach that would likely be far superior to our own capabilities.
On the fifth day another poster started a new thread that said the following.
Social Media Post: “Read the terms of service everyone!”…“as such the user consents to allow AI routines to filter for ambient avian noise at time periods that do not conform to the user's typical use schedule of the encompassing application. Collections made during these time periods will be bundled for distribution back to the project database during the next available submission window.”
All the technology in the world and the terms of service is still the place to go for when you want to do something legally without directly telling someone you plan to do something. It’s all on the up and up technically, but considering the average user sees a dozen to two dozen terms of service updates a week nowadays it’s the perfect place to hide features you want to run silently in the background. Though the language was cryptic, it was still pretty obvious. The application had been recording audio all day and night while clipping out segments with bird calls to be submitted when the next list is packaged to be delivered. The immediate implications of this were pretty broad. The Peregrine AI instances had been listening for birds most hours of the day for about 6 months on several hundred thousand phones. It seemed possible that the project had just collected more bird observations in that time frame than they had gotten directly from all users in the prior decade.
A further response on the social media thread asked the golden question that ultimately became the focal point of this entire situation.
Social Media Response: “Wait a minute. Peregrine grinding out birds when I’m not birding is one thing, but I have a question. How much better at audio id is Peregrine than I am at this point? Like, when I’m actually birding and Peregrine is helping me build a list or correcting my identifications, doesn’t that mean Peregrine's list is likely better than my list when it’s time to submit? I don’t recall Peregrine asking me if I heard any birds that it picked up in the background unless I’m actively using the sound id feature itself that was migrated over from Merlin. Am I no longer necessary in this equation?”
The final question sat in the digital space like an elephant in the back of a room nobody wanted to acknowledge. One commenter simply stated, “We are obsolete.”
An official statement and response did eventually come from the program coordinator. I’m not really sure even today whether it was good, bad, or just a slap in the face from reality. We learned that day the AI in our pocket along with the hardware it ran on was beyond anything we had ever imagined. It was so good in fact that developers realized it was capable of things no human could ever claim to be able to detect. Thus, they took some of the capabilities off line to prevent human list inflation from getting out of hand. They still wanted the data though so they decided to start it off by hiding that data to ease the ego bruises that were inevitable.
Official Statement From Peregrine Team: “We realize this response is overdue, but we wanted to make sure that we covered all the bases and answered as many questions as we could straight away. First we would like to offer an apology to our user base. It was not our intent to lie or mislead you in the process of getting to this point with the Peregrine app and AI core. The language found in the terms of service is of course required by our legal representation and we fully intended to tell everyone directly in the near future what the full capabilities of the Peregrine app really were. If you followed the AI response’s close enough it was always implying that it was recording audio even when you weren’t using the app, but we suspect that will come as little consolation.
The reason we didn’t immediately share this directly upon release of the app is that quite honestly we were skeptical that it was actually going to be this good. In the event that it proved out to be successful we wanted the hard data as proof of something that likely would be very difficult for enfranchised birders to stomach.
The reality is that Peregrine has, over the last 6 months, proven to be better at bird identification than 100% of application users during all time periods that those birders had been actively engaged in bird identification. This includes the fact that the bird watcher is able to count seen only birds that never make a noise in the wild. In fact, with any device running the newest microphone and chipset, Peregrine is currently able to hear and identify birds at distances that are physiologically impossible for humans. During testing we realized that reporting birds on screen or verbally to users they literally could never hear would create a problem. Users would for the first time be given evidence of a bird’s existence they might never be able to validate short of traveling to that bird. One might argue that would be a good thing and more birds would be found, but we ask that everyone please understand something. This wasn’t just happening once or twice during a birding session, it was happening non-stop the entire session.
If you imagine standing in the middle of a field with acres of grassland all around and listening for bird species that are making noise, you would inevitably reach a maximum number of birds present that you are able to detect. Taking a sample size of 1 hour our best field ornithologists peaked at 25 grassland species detected in the test location, but averaged closer to 17. Peregrine during the same timeframe detected 55 species because it was not only able to record the same 17 grassland species, but was able to add 3 additional in the grassland, 20 more species in a woodland a half mile away, 10 more to the north along a stream corridor, and finally 5 additional birds in a small cattail pond roughly ¾ of a mile away from the count position. During this same time period the best ornithologist detected a total of 42 individual birds while Peregrine detected 136 individuals.
It is not that Peregrine proved to be above average at voice identification of birds. The AI routines and audio cleaning features in place have shown to be superior in every way save for visual identification of birds that are silent. Repeated testing shows that in an environment with zero ambient noise save for those of mother nature, Peregrine is around 150% better by species count and in the neighborhood of 350% better at individual bird counts. These numbers do not account for an environment polluted with noise from highways, airplanes, wind, and other noises. In those spaces Peregrine excels in a way that is hard to imagine, often reaching species rates that are 800% above an advanced birder.
Our project and data collection faced a sobering reality. That our central mandate had reached a point where the primary value of the human involved in data collection was for visual identification and primarily as a vector for holding a recording device. In effect we have now trained a machine to be better than every human on earth at bird identification and we didn’t know what to do with that information beyond prove it in live testing. These last 6 months have created a firehose of information and if we just use the extra Peregrine AI submitted lists for that time period, while throwing out the user submitted lists, we would still have the equivalent of 10 years worth of collected data on birds.
All of the above combined means we are at a unique crossroads for bird data collection on the world-wide scale. Our legal team has informed us just this week that the sound identification AI core has been accepted by Google for inclusion in their standard mobile device deployment stack. This means that effective next month all mobile devices on the planet will receive an update to begin recording and automatically submitting bird population data. This may be a bit of a shock, but our project and all downstream dependent projects will have more data in a single day than has ever been submitted by users in the last 30+ years combined.
It may seem like the birding community is being abandoned and has been used, but we look at this as an achievement of everyone. Data collection will go mainstream and a normal everyday person equipped with a cell phone walking in a local park for exercise will now produce a bird list that previously depended on someone with decades of experience. Data consistency in these AI driven lists is so strong that most of our filtering technology to throw out inconsistent or inaccurate data can now be shelved and ignored. We will no longer need reviewers to engage users on flagged reports because the AI core will have handled such concerns.
For the time being the Peregrine app will continue to be downloadable and available for use. Our team will provide more information in the near future on the direction the app and our data collection efforts will go moving forward.”
——
It was what it was, as the kids used to say. We had just learned that the AI student had been trained on the decades of data we had collected and submitted and it was now a master. The tone of the statement indicated that we should expect some final technological stroke where we would have continued access to the technology and our lives would still be for the better when it comes to personal bird watching efforts, but that we should get comfortable with the fact that we would never have access to everything and that our own data was of relatively little value to science. For all intents and purposes we were not good enough or even smart enough to handle what was being produced. Even in the field and in the moment it would be a firehose in our face of bird identifications the likes of which we couldn’t handle or comprehend. For lack of a better phrasing we would have to evolve as humans before we could take advantage of newer and better technology.
