The Challenges in OSINT Analysis Concerning Digital Evidence

Introduction

Open-source intelligence (OSINT) has emerged as a critical tool for gathering information in today's digital world, utilising publicly available sources like social media, websites, and digital archives. However, the value of OSINT is not without its limitations, particularly when analysing digital evidence in legal and investigative settings. The authors, professionals with significant courtroom experience, raise issues which, when resolved, will be integral to overcoming these limitations, ranging from verification and context interpretation to data management and legal hurdles. This article is part of a broader series exploring unresolved issues surrounding OSINT and derived evidence. The main points explored in this article are directly relevant to the courtroom implications of these challenges, underscoring the weight of your responsibilities in these settings[1].

Verification and Reliability Issues

One of the primary challenges in OSINT analysis is ensuring the accuracy and reliability of the data. OSINT sources are often unverified, meaning the data can be outdated, incomplete, or misleading. This is particularly problematic when digital evidence is needed for legal or critical decision-making processes, where data integrity is paramount. The potential risks of false data, including incorrect conclusions and biased intelligence reports, underscore the importance of thoroughly verifying an OSINT source. Factors such as authority, accuracy, objectivity, timeliness, and relevance should be considered, along with the source's credibility and the ability to cross-verify information, to mitigate these risks[2][3].

Contextual Misinterpretation

Context is crucial when interpreting digital evidence. OSINT analysts and investigators, particularly those lacking subject-matter expertise or cultural understanding, may misinterpret data, leading to incorrect conclusions. Cultural nuances, language differences, and local event specifics can significantly impact how digital evidence should be understood. Therefore, a diverse team, including individuals with various cultural understandings, is beneficial and integral to ensuring comprehensive and accurate OSINT analysis. This diversity ensures that no critical elements are missed and the full significance of the evidence is grasped, resulting in unbiased and precise intelligence reports[4].

Data Overload and Analytical Challenges

The vast amount of data available through OSINT can overwhelm analysts and investigators, making it challenging to filter relevant information from the noise. This is where efficient data organisation plays a crucial role. Effective note-keeping and systematic data management are essential in managing this challenge. Analysts must determine which advanced tools can assist in efficiently organising and correlating data, enhancing the investigation's overall quality. However, the sheer volume of data can lead to information overload, where crucial details are buried under irrelevant content. Filtering this 'noise' is critical to identifying what data is valuable and what may be misinformation or contradiction[5][6].

Legal and Ethical Concerns

The collection and analysis of digital evidence through OSINT raise significant legal and ethical concerns. Even publicly available data may infringe on privacy rights or be subject to other legal restrictions. The lack of clear legal frameworks governing OSINT practices can lead to the use of illegally obtained information, which may be inadmissible in court. Ethical dilemmas arise when OSINT is used to surveil individuals or groups without their knowledge, potentially leading to power abuses or civil liberties violations [7].

One pertinent issue is the origins of the data and its ethical implications. For instance, organisations like Wikileaks[8] and incidents like the Panama Papers[9] highlight how leaked or stolen information can be widely disseminated and used, often by legitimate entities such as journalists or tax authorities. While the original access to such data may be illegal, subsequent access once published online, operates within legal and ethical grey areas, raising questions about the admissibility and morality of using such evidence in investigations[10].

Technological Limitations

Despite significant advances in OSINT tools, they have flaws. Many tools rely on algorithms to scrape and analyse data, but these algorithms can miss or misinterpret critical information due to technical limitations. For example, language-processing tools may need help with slang or dialects, and image recognition software might not correctly analyse poor-quality visuals. Additionally, specific data, such as content behind paywalls, encrypted communications, or data stored in private networks, may remain inaccessible to OSINT tools[11].

Moreover, the use of third-party online tools and applications introduces additional risks. These tools may store data remotely, raising concerns about data security and privacy. We think testing these tools regularly is essential to ensure they work as described and perform the stated tasks. However, the risks posed by outdated or unavailable tools emphasise the need for redundancy in OSINT processes, ensuring continuity and reliability.[12]

The Risk of Bias

Bias in OSINT analysis is another critical shortcoming. Data collected through OSINT may reflect the biases of those who create or share it, particularly on platforms like social media, where misinformation and echo chambers are standard. Analysts themselves may also impose their biases when selecting and interpreting data. This can lead to skewed analyses that fail to provide a balanced view of the evidence, potentially compromising the objectivity of the intelligence produced[13].

Integrating Digital Forensics with OSINT

Integrating digital forensics with OSINT offers a way to mitigate these challenges and significantly enhance investigative capabilities. Digital forensics, defined as the identification, preservation, and analysis of digital evidence following a legally accepted methodology, plays a crucial role in ensuring the accuracy and integrity of data. By using forensic techniques like metadata analysis, file integrity checks, and recovering hidden or deleted information[14], investigators can validate and strengthen the evidence gathered through OSINT.

Case studies demonstrate the practical benefits of this integration[15]. For instance, in cybersecurity investigations, combining OSINT with digital forensics has successfully identified and prosecuted cyber criminals by correlating publicly available information with forensic data from compromised systems. This synergy improves data correlation, enabling better resource allocation and more precise targeting during investigations[16].

However, this integration also presents challenges, such as maintaining the chain of custody and ensuring that devices used in analysis are not compromised or disclosed in formal proceedings[17]. Reliance on online tools that store data remotely further complicates matters, introducing potential data security and privacy risks[18].

Maintaining Evidential Integrity

One key challenge in integrating OSINT and digital forensics is preserving the integrity of the evidence throughout the investigation process. The entire chain of custody must be meticulously managed to ensure that evidence remains forensically sound and legally admissible. This involves adhering to established standards and guidelines, such as those outlined in the ACPO Guidelines[19] and NIST SP 800[20], which provide a structured methodology for handling digital evidence.

Furthermore, the origins of the evidence must be carefully considered, mainly when dealing with data obtained from open sources or leaked information. The ethical implications of using such data must be weighed against the legal requirements for admissibility, ensuring that the investigation remains within the bounds of the law while maintaining the highest standards of integrity[21].

Conclusion

While OSINT continues to be a powerful tool in intelligence and investigations, its limitations in handling digital evidence must be scrutinised, especially in legal contexts. From the hurdles of verification and context interpretation to the ethical and legal complexities, these challenges directly impact the integrity of evidence presented in court. Most professionals with extensive experience in legal proceedings will agree that integrating digital forensics and established evidence-handling methodologies with OSINT, coupled with rigorous analytical practices, is crucial to overcoming these obstacles. Doing so helps ensure that digital evidence meets the highest standards in courtrooms, preserving accuracy, admissibility and credibility.

OSINT professionals must continue contributing to the ongoing exploration of the many unresolved issues related to the admissibility of evidence, as integrity remains a paramount concern.

Authored by: The Coalition of Cyber Investigators

Paul Wright (United Kingdom) & Neal Ysart (Philippines)

© 2024 The Coalition of Cyber Investigators. All rights reserved.

The Coalition of Cyber Investigators is a collaboration between

Paul Wright (United Kingdom) - Experienced Cybercrime, Intelligence (OSINT & HUMINT) and Digital Forensics Investigator; and

Neal Ysart (Philippines) - Elite Investigator & Strategic Risk Advisor Advisor, Ex-Big 4 Forensic Leader.

With over 80 years of combined hands-on experience, Paul and Neal remain actively engaged in their field.

They established the Coalition to provide a platform to collaborate and share their experience and analysis of topical issues in the converging domains of investigations, digital forensics and OSINT. Recognizing that this convergence has created grey areas around critical topics, including the admissibility of evidence, process integrity, ethics, contextual analysis and validation, the coalition is Paul and Neal’s way of contributing to a discussion that is essential if the unresolved issues around OSINT derived evidence are to be addressed effectively. Please feel free to share this article and contribute your views.

[1] Open-source intelligence (OSINT): Techniques & Tools: Imperva (2023) Learning Center. Available at: https://www.imperva.com/learn/application-security/open-source-intelligence-osint (Accessed: September 08, 2024).

[2] Rid, T., & Buchanan, B. (2015). "Attributing Cyber Attacks." Journal of Strategic Studies, 38(1-2), 4-37. https://www.tandfonline.com/doi/abs/10.1080/01402390.2014.977382 (Accessed: September 08, 2024).

[3] OSINT vs Disinformation: The Information Threats ‘Arms Race.’ (n.d.). https://crestresearch.ac.uk/comment/osint-vs-disinformation-the-information-threats-arms-race (Accessed: September 08, 2024).

[4] Farwell, J. P., & Rohozinski, R. (2011). "Stuxnet and the Future of Cyber War." Survival, 53(1), 23-40. https://www.tandfonline.com/doi/abs/10.1080/00396338.2011.555586 (Accessed: September 08, 2024).

[5] McAfee Institute (no date) 'Leveraging advanced OSINT techniques for enhanced investigative outcomes,' McAfee Institute. https://blog.mcafeeinstitute.com/articles/leveraging-advanced-osint-techniques-for-enhanced-investigative-outcomes (Accessed September 08, 2024).

[6] Efim and Efim (2024) 10 Advanced OSINT techniques for effective investigations. https://espysys.com/blog/10-advanced-osint-techniques/ (Accessed September 08, 2024).

[7] Media Sonar (2022) OSINT Techniques | Legal & Ethical of Open Source Intelligence. https://mediasonar.com/2020/04/30/legal-ethical-osint/ (Accessed September 08, 2024).

[8] WikiLeaks. (2019) Wikileaks.org website: https://wikileaks.org/-Leaks-.html (Accessed September 08, 2024).

[9] ICIJ Offshore Leaks Database. (2020) Icij.org website: https://offshoreleaks.icij.org (Accessed September 08, 2024).

[10] Brandefense. (n.d.). Top Open Source Intelligence (OSINT) tools for the dark web. Brandefense https://brandefense.io/blog/dark-web/top-open-source-intelligence-osint-tools-for-dark-web/ (Accessed September 08, 2024).

[11] The importance of recognising biases in protective Intelligence Analysis | Allied Universal. (n.d.). https://www.aus.com/blog/importance-recognizing-biases-protective-intelligence-analysis (Accessed September 08, 2024).

[12] Owl, G. (2023) OSINT vs. Cyber Forensics: Understanding the Synergy in Cybersecurity. https://www.linkedin.com/pulse/osint-vs-cyber-forensics-understanding-synergy-cybersecurity-s6etf/ (Accessed September 08, 2024).

[13] Mandia, K., & Prosise, C. (2014). "Incident Response and Computer Forensics." McGraw-Hill Education. https://dl.acm.org/doi/abs/10.5555/2825951 (Accessed September 08, 2024).

[14] (Altlaw. (2024, January 15). Digital Forensics Best Practices: Evidence Collection Steps2024 https://www.altlaw.co.uk/blog/digital-forensics-best-practices-5-essential-steps-in-the-evidence-collection-process (Accessed September 08, 2024).

[15] Lerner, E. (2024) Digital Forensics and OSINT: Synergy in Modern Investigations. https://www.linkedin.com/pulse/digital-forensics-osint-synergy-modern-investigations-efim-lerner-aw8lf/ (Accessed September 08, 2024).

[16] Lerner, E. (2024) Digital Forensics and OSINT: Synergy in Modern Investigations. https://www.linkedin.com/pulse/digital-forensics-osint-synergy-modern-investigations-efim-lerner-aw8lf/ (Accessed September 08, 2024)

[17] Clarke, S. (2024, January 12). How to Stay Secure During your OSINT Investigation: Security, Privacy and OSINT in 2024. Blackdot Solutions Videris. https://blackdotsolutions.com/blog/security-privacy-and-osint (Accessed September 08, 2024).

[18] Lee, H. (2024) 'The crucial role of chain of custody: Ensuring evidence integrity and quality assurance | Evidence,' Evidence Management Institute, 21 May. https://evidencemanagement.com/the-crucial-role-of-chain-of-custody-ensuring-evidence-integrity-and-quality-assurance/ (Accessed September 08, 2024).

[19] ACPO Good Practice Guide for Digital Evidence (2012). https://www.digital-detective.net/digital-forensics-documents/ACPO_Good_Practice_Guide_for_Digital_Evidence_v5.pdf (Accessed September 08, 2024).

[20] NIST SP 800-86 Standards as a Framework for Digital Forensic Evidence (2022) https://nvlpubs.nist.gov/nistpubs/ir/2022/NIST.IR.8428.pdf (Accessed September 08, 2024).

[21] Efim and Efim (2024) The Power of OSINT in Digital Forensics: Tapping into Publicly Available Data. https://espysys.com/blog/osint-in-digital-forensics/ (Accessed September 08, 2024).