Ghosts in the data: false detections in VEMCO pulse position modulation acoustic telemetry monitoring equipment
© Simpfendorfer et al. 2015
Received: 1 July 2015
Accepted: 11 December 2015
Published: 29 December 2015
False-positive data (better known as “false detections”) in VEMCO VR2 acoustic telemetry monitoring studies that use pulse position modulation coding can cause biased or erroneous outcomes in data analysis. To understand the occurrence of false detections in acoustic monitoring data sets, the results of a range test experiment using eight acoustic receivers and 12 transmitters were examined.
One hundred and fifty one tag ID codes were recorded, 137 of which were identified as likely from false detections, 12 were from test tags, and two were from tagged sharks. False detections accounted for <0.05 % of detections (918) in the experiment. False detection tag ID codes were not randomly distributed amongst the available codes, being more likely to occur at IDs close to tags used in the experiment. Receivers located near the bottom recorded the most false detections and tag ID codes from false detections. Receivers at the same depth did not differ significantly in the mean number of daily false detections. The daily number of false detections recorded by a receiver did not conform to a random pattern, and was not strongly correlated with daily receiver performance.
In an era of increasing data sharing and public storage of scientific data, the occurrence of false detections is of significant concern and the results of this study demonstrate that while rare they do occur and can be identified and accounted for in analyses.
Whenever false positives occur in ecological data, they have the potential to cause significant effects on the interpretation of results . In some areas of ecology, such as automated acoustic monitoring of amphibian, bird, cetacean, and bat sounds, issues related to false-positive data have been assessed in detail [2–4] and their effects on results of analyses, such as species distribution models, have been accounted for . In other ecological fields, false-positive data have been poorly examined. One such field is that of aquatic acoustic telemetry monitoring of tagged animals, herein referred to as acoustic monitoring.
Acoustic monitoring has become a popular tool in the study of aquatic animals as it enables a large number of subjects to be tracked simultaneously [5–7]. While there are several equipment manufacturers or standards, the most commonly used is the VEMCO VR2 system which uses a proprietary pulse position modulation coding scheme. This VR2 system is a two-part approach: acoustic receivers are deployed in a study area to listen for coded signals from transmitters deployed on released animals. Each transmitter produces a signal with a unique tag ID code. The number of tag ID codes available to users depends on the coding scheme used, with multiple code spaces (i.e. coding schemes that define the number of available tag ID codes than can be encoded) available to increase the number of unique tag ID codes and allow the provision of different forms of telemetry data, e.g. temperature, depth, and acceleration. Animal detection data from receivers are used to generate information on movements, residency, behaviour, and many other ecological aspects .
Acoustic monitoring researchers have long known about the occurrence of false-positive data (known more commonly as ‘false detections’) in their datasets. For example, Heupel et al.  excluded detections of known tags using a speed criteria in data from grey reef sharks (Carcharhinus amblyrhynchos) that had potentially moved long distances (>100 km) between two acoustic arrays. The importance of false detections has been commonly overlooked since research primarily focuses on known tag ID codes from animals deployed in relatively small areas. As acoustic monitoring increases in popularity and data sharing grows amongst research groups through the implementation of regional, national, and international databases of detections (e.g. the Atlantic Cooperative Telemetry Network—http://www.theactnetwork.com/home, the Australian Animal Tracking and Monitoring System—https://aatams.emii.org.au/aatams, and the Ocean Tracking Network—http://oceantrackingnetwork.org), the potential for false detections to arise in datasets drastically increases as does the potential for erroneous interpretation. It therefore is becoming increasingly important to understand the nature of false detections and the processes that create them and therefore identify which tag ID codes are relevant to analyses.
False detections occur when transmissions from two or more transmitters collide, which results in a detection of a different tag ID code by the receiver . Within a dataset two situations can occur. First, a false detection can produce an unknown tag ID code, i.e. a code from a transmitter that has not been released in a specific study location (but may have been released elsewhere), herein referred to as a Type A false detection. For researchers using data from their own study area, these Type A false detections are relatively easy to identify and discard from analyses because the unique tag ID codes erroneously created differ from those released. However, in shared datasets these are much more difficult to identify. Alternatively, a false detection may result in a tag ID code that is the same as the one from a transmitter that has been released; herein referred to as a Type B false detection. Differentiating Type B false positives from true-positive data is inherently more difficult. Importantly, shared data may include Type A false detections that are difficult to identify without the broader information and context derived from the local study and experimental design. Thus, without adequate quality control algorithms false detections may be used in data analyses and lead to biased or erroneous outcomes.
To better understand the occurrence of false detections in acoustic monitoring data, we used detection data from a range test experiment carried out over a 9-month period. This experiment allowed for the examination of false detections in a controlled situation. The specific aims were to (1) determine the occurrence of Type A and Type B false detections, (2) examine the consistency in the occurrence of Type A false detections between receivers, (3) determine if there were patterns in the tag ID codes of Type A false detections, (4) examine the pattern of Type A false detections over time, and (5) examine if false detections occurred as a result of random processes or if they were correlated with the performance of receivers.
A total of 151 unique tag ID codes were recorded by the eight receivers during 235 days of monitoring. These tag codes resulted in 1 975 165 detections. Twelve of the tag ID codes were from tags deployed for the range test experiment and two were from tags registered in the AATAMS database: one white shark (Carcharodon carcharias) and one wobbegong shark (Orectolobus maculatus). The remaining 137 tag ID codes were from no identifiable source and were potentially the result of Type A false detections. All of the known tag ID codes were identified by the VUE False Detection Analyser as likely to be from valid tags (experiment tags: 96 tag ID—receiver combinations; white shark: eight tag ID—receiver combinations; and wobbegong shark: one tag ID—receiver combination), while all of the unknown codes were identified as likely to come from Type A false detections (384 tag ID—receiver combinations). The tag ID codes that were identified as likely from false detections accounted for 918 detections (<0.05 % of detections) throughout the study period. The majority of the 137 tag ID codes (98.5 %) identified as Type A false detections were from the A69-1303 code space, i.e. the same as all of the experiment tags and the two detected species. Single tag ID codes from the A69-1105 and A69-1008 code spaces were also detected. Examination of time differences between detections of the experimental tags did not identify any Type B false detections, with all time differences greater than or equal to the specified repeat rate.
Distribution of false detection tag ID codes by receiver
Number of false detection tag ID codes
Number of unique false detection tag ID codes
Detection of false tag ID codes
Mean number of false detections per day
Comparison to Poisson (random daily detections)
Results of pairwise Wilcoxon rank-sum tests comparing mean numbers of daily Type A false detections between receivers
Correlation coefficient values between daily numbers of Type A false detections and receiver performance metrics
Code detection efficiency
The results of this study demonstrate that false detections can occur in acoustic monitoring studies using the VEMCO VR2 system, but that their occurrence is rare relative to the number of detections of valid tags (<0.05 % of detections). Only Type A false detections were identified, suggesting that Type B false detections are much less common in acoustic monitoring datasets. It is possible that Type B false detections did occur but could not be identified by looking for sequential detections at shorter intervals than the repeat rate of the transmitters. Other studies have identified Type B false detections (e.g. 8) using limits on animal movement ability. While such an approach could not be used for this study because the transmitters were stationary, it may be particularly useful in studies that combine data from receivers placed large distances apart. The ability of the VEMCO False Detection Analyser to correctly identify known transmitters as likely valid and unknown tags as likely invalid indicates that this approach is suitable for identifying Type A false detections in datasets.
The placement of receivers in the water column appears to have an effect on the number of false detections, the number of false detection tag ID codes, and the consistency through time with which new tag ID codes were discovered. The near-bottom receivers, and especially the downward facing ones, performed worst in all of the false detection metrics examined. This may be the result of increased likelihood of reflection of sound waves from the bottom reaching these receivers and so increasing the likelihood of two signal transmissions colliding. It may also be possible that the reflection of a single transmission may interfere with itself resulting in a false detection. Environmental noise can also be a factor effecting acoustic detections . Receivers at different levels in the water column detected similar numbers of false detections, although they did not necessarily detect the same tag ID codes. Differences in tag ID codes between receivers with similar placement suggest that tag collision events that lead to false detections are interpreted differently even between closely located units. The greater differences in the number of false detections between receivers with different placement may suggest that relatively small changes in distance from the transmitters of the colliding signals affect the tag ID code recorded. The data also suggested that receivers further from the bottom produced fewer Type A false detections. This may be the result of increased reflection of sound and biological noise closer to the bottom. These results provide clear guidance for receiver placement to acoustic monitoring practitioners wishing to minimise the occurrence of false detections in their data. Receivers further from the bottom are likely to produce less false detections, and if they are placed close to the bottom, then they should face upwards. However, researchers must weigh this advantage against the ability of the receivers to detect the tagged animals.
The repeat rate of the experimental transmitters (the period between signal transmissions by the tag) used in this study was substantially longer than that used by most researchers in the field. The long repeat rate was used deliberately in this experiment to reduce the likelihood of tag collisions affecting range testing. This suggests that for many animal-based studies that routinely use transmitters with faster repeat rates, there may be more false detections than observed in this study. Whether this would result in more tag ID codes from Type A false detections, greater numbers of a similar set of tag ID codes, or both, remains to be investigated. The lack of asymptotes in the tag ID code discovery curves suggest that more codes would have been detected in this study if a shorter repeat rate had been used. Faster repeat rates are also likely to increase the probability of Type B false detections and further work to identify this type of false detection is needed.
The processes that lead to false detections suggest that they should occur randomly through time. However, in this study the number of Type A false detections recorded each day was not random, suggesting that other processes may affect their occurrence. We were able to exclude changes in the performance of receivers  as factors that affected the numbers of Type A false detections. Further investigation will be required to identify the processes that affect the occurrence of false detections, including changes in environmental conditions and biofouling. [10, 11].
The distribution of Type A false detection tag ID codes within the code space used in this study was also not randomly distributed, with codes more likely to occur close to real tag ID codes in the system and also occurring in groups. This suggests that the collisions between pairs (or more) of real tag ID codes can only create a subset of false detection codes because of the way that codes are transmitted. This hypothesis was supported by the curvature in the tag ID discovery curves from near-surface and mid-water receivers that suggested that few new codes were being recorded, while only a small proportion of possible tag ID codes were being detected. This situation may also have been exacerbated by the stationary nature of this study, and as such the distribution of false detection codes may be different in studies where animals are moving relative to receivers.
Studies examining false detections in acoustic monitoring data are limited, and there is a need to further understand their occurrence and the factors that affect their prevalence. In an age of increasing data sharing and public storage of scientific data, these issues are of significant concern. Ensuring that the data are quality controlled and used within the context of the experimental design parameters is crucial to avoiding erroneous conclusions. This study demonstrated that there are patterns in the Type A false detection tag ID codes and their occurrence was not always random. It will be important for researchers to understand the occurrence of false detections within datasets, especially where data are obtained from sources other than their own receivers. In these situations, it will be critical to employ checks for false detections to ensure that only valid data are used.
Data from each of the receivers were downloaded into a VEMCO VUE database from which information on the number of detections of individual tag ID codes by individual receiver and receiver performance was obtained. In addition, the False Detection Analyser function in VUE version 2.1.3 was used to evaluate the likelihood that individual tag ID codes detected on individual receivers (i.e. false detections were analysed for each tag ID at each receiver, referred to as a tag ID–receiver combination) were the result of false detections. The False Detection Analyser used an algorithm that determines the time between detections of a tag ID code on each receiver and computes the ratio of short periods between detections (default value <30 min) to long periods between detections (default value >12 h) . Only the default values were used in the False Detection Analyser. If there were more long periods between detections than short periods, then a tag ID code was considered to have a high likelihood of being the result of false detections; otherwise detections from tag ID codes were considered to be valid. The False Detection Analyser could not be used for tag ID codes that had a single detection and these were assumed to have a high likelihood of being the result of a false detection.
We distinguished between two types of false detections: false detections that generated tag ID codes that are unknown (Type A) and those that resulted in tag ID codes from known tags (Type B). To determine if false detections produced tag ID codes that were the same as those from the experimental tags (Type B), we examined the data from each receiver for detections of known tags at less than the specified repeat rate (550–650 s) by calculating the difference in time between detections for each receiver and experimental tag. The values of time differences less than 520 s (a 30-s buffer period was added to the shortest specified repeat rate to ensure that there were clear differences from the repeat rate) indicated likely false detections of known tags. This approach could not identify Type B false detections that occurred at times longer than the repeat rate and thus were considered a conservative estimate.
A data set that contained the number of detections of each tag ID code, receiver, and day was generated for analysis. Tag ID codes that were known from the experiment and any others that were included in the AATAMS database (https://aatams.emii.org.au/aatams) or were identified by the False Detection Analyser as “likely valid” tags were removed from this data set. This filtering process left a data set that contained only those tag ID codes that were likely the result of Type A false detections. To determine if the tag ID codes in this data set occurred randomly throughout the A69-1303 code space, the tag ID code numbers were used in a Wald–Wolfowitz runs test. To test if tag ID codes from false detections occurred closer to the numeric tag ID codes used in the experiment, i.e. close in number sequence, the absolute value of the difference between the false detection code numbers and the lowest experimental code number was calculated (each tag ID code includes a unique code number; in this case between 1 and 65,536). To determine if the rate of adding new Type A false detection tag ID codes decreased over time, discovery curves were constructed by plotting the number of unique tag codes recorded from the start of the study to each of the 232 days following deployment. Discovery curves that were straight indicated a consistent rate of adding new tag ID codes, while those that reach an asymptote indicate that no new tag IDs were detected toward the end of the experiment.
To determine if the distributions of the daily number of receivers on which false detections occurred and if the daily number of detections on a single receiver were the result of random processes, they were compared to a Poisson distribution using a chi-squared goodness-of-fit test. The mean numbers of daily false detections were compared between pairs of receivers using Wilcoxon rank-sum tests since the data were not normally distributed. A Bonferroni correction was applied to the critical probability value for this test because 28 comparisons were made using the same dataset.
Daily performance metrics for each acoustic receiver were calculated following Simpfendorfer et al. . Five metrics were used: the number of pings (the number of acoustic pings recorded at the receiver’s working frequency; an A69-1303 tag ID code is made up of eight pings), number of detections, code detection efficiency (the ratio of detections to synchronisation codes), rejection coefficient (number of codes rejected due to an invalid checksum divided by the number of synchronisation codes), and noise quotient (the number of pings received over and above those that resulted from tag detections). The correlation coefficient between daily values of the number of false detections and values of performance metrics was calculated to determine if numbers of false detections were related to receiver performance or factors that also affected receiver performance.
CS helped design the experiment, analysed data, and wrote the manuscript. CH designed the experiment, analysed data, and helped write the manuscript. AS analysed data and helped write the manuscript. KT analysed data and helped write the manuscript. XH analysed data and helped write the manuscript. RH designed the experiment and helped write the manuscript. MH helped design the experiment, analysed data, and helped write the manuscript. All authors read and approved the final manuscript.
This experiment was conducted under the auspices of the Australian Animal Tagging and Monitoring System (AATAMS) Scientific Committee. AATAMS is part of the Integrated Marine Observing System (IMOS) supported by the Australian Government through the National Collaborative Research Infrastructure Strategy and the Super Science Initiative. We thank D Webber and H Pederson from VEMCO for technical information and advice.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Chambert T, Miller DAW, Nichols JD. Modeling false positive detections in species occurrence data under different study designs. Ecology 2015;96:332–9. doi:10.1890/14-1507.1.PubMedView ArticleGoogle Scholar
- Marques TA, Thomas L, Ward J, DiMarzio N, Tyack PL. Estimating cetacean population density using fixed passive acoustic sensors: an example with Blainville’s beaked whales. J Acoust Soc Am. 2009;125(4):1982–94. doi:10.1121/1.3089590.PubMedView ArticleGoogle Scholar
- Bardeli R, Wolff D, Kurth F, Koch M, Tauchert KH, Frommolt KH. Detecting bird sounds in a complex acoustic environment and application to bioacoustic monitoring. Pat Rec Lett. 2010;31(12):1524–34. doi:10.1016/j.patrec.2009.09.014.View ArticleGoogle Scholar
- Blumstein DT, Mennill DJ, Clemins P, Girod L, Yao K, Patricelli G, et al. Acoustic monitoring in terrestrial environments using microphone arrays: applications, technological considerations and prospectus. J Appl Ecol. 2011;48(3):758–67.View ArticleGoogle Scholar
- Heupel MR, Semmens JM, Hobday AJ. Automated acoustic tracking of aquatic animals: scales, design and deployment of listening station arrays. Mar Freshw Res. 2006;57(1):1–13.View ArticleGoogle Scholar
- Heupel MR, Webber DM. Trends in acoustic tracking: where are the fish going and how will we follow them?. In: McKenzie J, Parsons B, Seitz A, Kopf RK, Mesa M, Phelps Q, editors. Advances in Fish Tagging and Marking Technology. Bethesda: American Fisheries Society; 2012. p. 219–31.Google Scholar
- Voegeli FA, Smale MJ, Webber DM, Andrade Y, O’Dor RK. Ultrasonic telemetry, tracking and automated monitoring technology for sharks. Env Biol Fish. 2001;60(1–3):267–81.View ArticleGoogle Scholar
- Heupel MR, Simpfendorfer CA, Fitzpatrick R. Large-scale movement and reef fidelity of grey reef sharks. PLoS ONE. 2010;5(3):e9650.PubMedPubMed CentralView ArticleGoogle Scholar
- Pincock DG. False detections: what they are and how to remove them from detection data. Amirix Document DOC-004691 Version 03. 2012; Available: http://vemco.com/wp-content/uploads/2012/11/false_detections.pdf.
- Udyawer V, Chin A, Knip DM, Simpfendorfer CA, Heupel MR. Variable response of coastal sharks to severe tropical storms: environmental cues and changes in space use. Mar Ecol Prog Ser. 2013;480:171–83.View ArticleGoogle Scholar
- Heupel MR, Reiss KL, Yeiser BG, Simpfendorfer CA. Effects of biofouling on performance of moored data logging acoustic receivers. Limn Ocean-Meth. 2008;6:327–35.View ArticleGoogle Scholar
- Simpfendorfer CA, Heupel MR, Collins AB. Variation in the performance of acoustic receivers and its implication for positioning algorithms in a riverine setting. Can J Fish Aq Sci. 2008;65(3):482–92.View ArticleGoogle Scholar