| Title: |
Anomaly Detection and Approximate Similarity Searches of Transients in Real-time Data Streams |
| Authors: |
Aleo, PD; Engel, AW; Narayan, G; Angus, CR; Malanchev, K; Auchettl, K; Baldassare, VF; Berres, A; de Boer, TJL; Boyd, BM; Chambers, KC; Davis, KW; Esquivel, N; Farias, D; Foley, RJ; Gagliano, A; Gall, C; Gao, H; Gomez, S; Grayling, M; Jones, DO; Lin, C-C; Magnier, EA; Mandel, KS; Matheson, T; Raimundo, SI; Shah, VG; Soraisam, MD; de Soto, KM; Vicencio, S; Villar, VA; Wainscoat, RJ |
| Publisher Information: |
American Astronomical Society; Department of Pure Mathematics and Mathematical Statistics; //doi.org/10.3847/1538-4357/ad6869 |
| Publication Year: |
2024 |
| Collection: |
Apollo - University of Cambridge Repository |
| Subject Terms: |
5101 Astronomical Sciences; 51 Physical Sciences |
| Description: |
We present Lightcurve Anomaly Identification and Similarity Search (LAISS), an automated pipeline to detect anomalous astrophysical transients in real-time data streams. We deploy our anomaly detection model on the nightly Zwicky Transient Facility (ZTF) Alert Stream via the ANTARES broker, identifying a manageable ∼1–5 candidates per night for expert vetting and coordinating follow-up observations. Our method leverages statistical light-curve and contextual host galaxy features within a random forest classifier, tagging transients of rare classes (spectroscopic anomalies), of uncommon host galaxy environments (contextual anomalies), and of peculiar or interaction-powered phenomena (behavioral anomalies). Moreover, we demonstrate the power of a low-latency (∼ms) approximate similarity search method to find transient analogs with similar light-curve evolution and host galaxy environments. We use analogs for data-driven discovery, characterization, (re)classification, and imputation in retrospective and real-time searches. To date, we have identified ∼50 previously known and previously missed rare transients from real-time and retrospective searches, including but not limited to superluminous supernovae (SLSNe), tidal disruption events, SNe IIn, SNe IIb, SNe I-CSM, SNe Ia-91bg-like, SNe Ib, SNe Ic, SNe Ic-BL, and M31 novae. Lastly, we report the discovery of 325 total transients, all observed between 2018 and 2021 and absent from public catalogs (∼1% of all ZTF Astronomical Transient reports to the Transient Name Server through 2021). These methods enable a systematic approach to finding the “needle in the haystack” in large-volume data streams. Because of its integration with the ANTARES broker, LAISS is built to detect exciting transients in Rubin data. |
| Document Type: |
article in journal/newspaper |
| File Description: |
application/pdf |
| Language: |
English |
| Relation: |
https://www.repository.cam.ac.uk/handle/1810/375082 |
| Availability: |
https://www.repository.cam.ac.uk/handle/1810/375082 |
| Rights: |
Attribution 4.0 International (CC BY 4.0) ; https://creativecommons.org/licenses/by/4.0/ |
| Accession Number: |
edsbas.78107E6F |
| Database: |
BASE |