Category Archives: Information Management

De-NISTing in eDiscovery: A Costly Provision That Shouldn’t Be in Model Orders in the First Place

By John Patzakis

A model eDiscovery order I recently came across from a federal district court issued by a respected judge included a provision requiring parties to de-NIST their files in the course of eDiscovery production. On its face, this may seem like a reasonable technical requirement to some practitioners. But this provision reflects a fundamental misunderstanding of how proportional, targeted eDiscovery collection should work — and it points to a broader problem in our industry that deserves some attention.

For those unfamiliar with the term, de-NISTing refers to the process of filtering out known, irrelevant system files from a forensic collection using the National Institute of Standards and Technology’s reference database of known file signatures. The NIST database catalogs hundreds of thousands of known operating system files, executables, DLL files, and other system-generated data that have no evidentiary value whatsoever. De-NISTing removes these files from a collection so that reviewers are not burdened with wading through mountains of irrelevant system data. The reason you need to de-NIST in the first place is because you collected a full-disk image — capturing everything on the drive, relevant or not.

And that is precisely the problem with requiring de-NISTing in a model eDiscovery order. As I have written extensively, including in our recent white paper on proportionality in eDiscovery, courts have consistently held that full-disk imaging is not the appropriate default for civil litigation collections. Going all the way back to Deipenhorst v. City of Battle Creek in 2006, courts have warned that imaging a hard drive results in the production of massive amounts of irrelevant — and potentially privileged — information. More recently, in Motorola Solutions v. Hytera Communications Corp., the court emphasized that forensic examination of a party’s computers “is no routine matter” and that courts must use caution to avoid unduly impinging on privacy interests. A model order that presupposes full-disk imaging by requiring de-NISTing is, at minimum, inconsistent with this well-established body of case law.

The 2015 amendments to Federal Rule of Civil Procedure 26(b)(1) established a clear six-pronged proportionality framework for eDiscovery, requiring parties and courts to weigh factors including the importance of the issues at stake, the amount in controversy, the parties’ resources, and whether the burden or expense of proposed discovery outweighs its likely benefits. Courts have taken these amendments seriously and have consistently limited overbroad discovery requests on proportionality grounds. A blanket model order requirement to de-NIST implicitly endorses a collect-everything methodology that runs counter to the proportionality principles embedded in Rule 26(b)(1) and the extensive case law that has developed around it.

So how does a provision like this end up in a model court order? The answer, I believe, lies in the undue influence that certain eDiscovery service providers have had on collection practices and, ultimately, on the drafting of court orders and guidelines. Some service providers have a clear financial incentive to collect as much data as possible, since their fees are calculated on a per-gigabyte basis — meaning the more data collected, processed, and hosted, the higher the bill. This volume-based business model has shaped industry “best practices” in ways that favor over-collection, and that mindset has quietly seeped into the thinking of some federal judges and the model orders they issue. What gets dressed up as technical diligence is, in many cases, simply an artifact of a business model that profits from excess.

If you are conducting a properly scoped, targeted eDiscovery collection that is consistent with the principles of proportionality — as the Federal Rules and overwhelming case law require — there is simply no reason to de-NIST. A targeted collection does not reach system files, executables, DLLs, or other non-user-generated data in the first place. You are collecting potentially relevant ESI from identified custodians, scoped by search terms, date ranges, file types, and data sources. You never touch the data that de-NISTing is designed to filter out, which means the entire de-NISTing step — and its associated cost and processing time — is unnecessary overhead born entirely of an overbroad collection methodology.

This is precisely the approach built into X1 Enterprise, which enables legal and IT teams to conduct targeted, remote collections across large numbers of custodians without ever capturing the system-level data that necessitates de-NISTing. X1 Enterprise collects only the user-generated, potentially relevant ESI within defined parameters, preserving full metadata integrity and maintaining a documented chain of custody — satisfying every requirement for forensic soundness without the bloat, expense, and proportionality concerns of full-disk imaging. In an era where courts are increasingly scrutinizing eDiscovery costs and demanding proportionality, practitioners and judges alike should be asking not how to manage the mess created by over-collection, but how to avoid creating that mess in the first place.

Leave a comment

Filed under Best Practices, Case Law, Cloud Data, Cybersecurity, Data Audit, Data Governance, eDiscovery, eDiscovery & Compliance, Enterprise eDiscovery, GDPR, Information Governance, Information Management

Why Most Tools Fall Short for Large-Scale Information Governance and What Actually Works

By John Patzakis

For more than a decade, enterprise organizations have struggled with a persistent and costly challenge: how to effectively search, collect, manage, and analyze large volumes of unstructured on-premise data for information governance, eDiscovery, and enterprise search use cases. We are talking about environments with many terabytes of data distributed across file servers, email archives, endpoints, and Microsoft 365 data that must be rapidly interrogated, precisely analyzed, and in many cases urgently remediated in response to a regulatory inquiry, a data breach, or an M&A transaction. Despite the proliferation of tools claiming to address this challenge, none has ever truly solved it at scale. The core reason is architectural. Most of these tools are built on a flawed foundation from the start.

The gravitational pull toward Elasticsearch as the search foundation for enterprise data tools is easy to understand. It is open source, it is widely documented, and it is written in Java a language familiar to a large pool of developers. For these reasons, a basic centralized search and analysis tool can be assembled relatively quickly, and hundreds of vendors and in-house development teams have taken exactly this path. The problem is not that Elasticsearch lacks capability for general-purpose search. The problem is that general-purpose search and large-scale enterprise information governance are fundamentally different problems, and what works for one fails badly at the other. What is rarely discussed openly but what practitioners learn the hard way is that Elasticsearch’s architectural limitations are not configuration issues that can be engineered around. They are structural constraints baked into the platform’s design, and they surface precisely at the scale and complexity that serious information governance work demands.

The result is a graveyard of failed or severely limited information governance deployments: tools that work impressively in demos on curated datasets of a few hundred gigabytes, but that buckle, stall, or simply break when asked to operate on the multi-terabyte, distributed, live data environments that characterize real enterprise compliance projects.

The Structural Limitations of Elasticsearch for Information Governance
The memory problem with Elasticsearch begins with Java itself, which requires a significant amount of compute power over other code bases when addressing large volumes of data. The Java Virtual Machine (JVM) requires a heap to manage object allocation, and as data volumes grow, the memory demands scale dramatically. Each Elasticsearch index must be loaded into memory to be searched, and in a multi-terabyte environment with complex query patterns — the kind that information governance work consistently requires — the JVM heap pressure becomes severe and unmanageable. Organizations that have attempted to deploy Elasticsearch-based platforms against over 10 terabytes of enterprise data consistently encounter the same outcome: massive hardware requirements, constant tuning, and performance that degrades as the dataset grows rather than holding steady. The compute overhead is not a solvable problem; it is an inherent consequence of building a memory-intensive centralized index on a Java runtime, and it places a practical ceiling on what Elasticsearch-based governance tools can realistically accomplish.

Beyond the memory constraints, the workflow required to use Elasticsearch for information governance introduces a second, equally serious problem: it requires a full copy of the data under governance to be made and migrated into the centralized index. For a 50-terabyte dataset, this means creating 50 additional terabytes of sensitive material — often including personally identifiable information, privileged communications, and confidential business records — and transferring it outside its original, controlled location. Requiring the wholesale copying and centralization of that same data in order to govern it is a fundamental contradiction, one that legal, security, and compliance stakeholders increasingly and rightly reject.

The timeline problem compounds the data duplication problem. Copying, transferring, and indexing 50 terabytes of enterprise data into a centralized Elasticsearch platform is not a weekend project. In real-world deployments, this process can take months, even under favorable conditions. And information governance use cases are rarely patient ones. Data breach impact assessments operate under regulatory notification deadlines measured in days. M&A-related data audits run on compressed timelines driven by transaction closing schedules. By the time the data has been staged and indexed into a centralized Elasticsearch platform, the underlying data has changed, and the copied index set is already stale.

Finally, even if an organization tolerates the data duplication, survives the timeline, and manages the memory overhead, there is a “last mile” problem that the centralized Elasticsearch architecture cannot solve: remediation. Information governance is not just about finding sensitive or problematic data — it is about acting on it — Deleting records past their retention period. Quarantining compromised PII. Tagging and separating data in support of a corporate divestiture. When the discovery and analysis workflow is built on a centralized copy of the data, the organization is operating on clones, not originals. The identified data still exists in its original locations distributed across file servers, Microsoft 365 environments, laptops, and cloud storage. Tracing back from a finding in a centralized index to the live source, and then executing a remediation action on that source, is a manual, error-prone, and operationally disruptive process.

How X1 Enterprise’s Micro-Indexing Architecture Solves What Elasticsearch Based Tools Cannot
X1 Enterprise is built on a fundamentally different architectural premise: rather than requiring data to be copied and centralized, X1’s patented micro-indexing technology indexes, searches, analyzes, and remediates data entirely in place where it lives, within the corporate environment, without ever moving it. This architectural difference is consequential at every stage of a large-scale governance project. The micro-indexing engine is written in C++, which delivers dramatically more efficient memory utilization than a Java-based runtime. Individual micro-indexes do not need to be loaded into memory simultaneously; the architecture is genuinely distributed and parallelized, enabling X1 Enterprise to operate effectively at multi-terabyte scale, including at hundreds of terabytes, without the memory walls and hardware escalation that make Elasticsearch-based platforms impractical for serious enterprise deployments.

Because X1 Enterprise operates in place, the data duplication problem is eliminated entirely. There is no second copy of your sensitive data to govern, secure, or explain to regulators. The indexed data remains in its original location, under the organization’s existing controls, throughout the entire governance workflow. This means that X1 Enterprise not only avoids compounding compliance risk, it actively reduces it, by ensuring that sensitive data never leaves its controlled environment. For organizations subject to GDPR, HIPAA, CCPA, or sector-specific data residency requirements, the ability to conduct large-scale information governance analysis entirely within the corporate firewall is not a luxury. It is a hard requirement. X1 Enterprise is the only platform in the market that can meet this requirement at multi-terabyte scale without architectural compromise.

Perhaps most powerfully, the in-place architecture closes the remediation loop that Elasticsearch-based tools leave permanently open. When X1 Enterprise identifies data that must be deleted, preserved, tagged, or acted upon, it can execute that remediation directly on the source data in Microsoft 365, on file servers, on endpoints, wherever the data resides. There is no manual tracing back from a centralized index to a distributed original. The finding and the action occur in the same environment, with full auditability and chain-of-custody documentation.

X1 Enterprise delivers the architecture that the industry has needed for years.

To learn more, schedule a briefing today at sales@x1.com or visit x1.com/solutions/x1-enterprise-platform.

Leave a comment

Filed under Best Practices, Business Productivity Search, Data Governance, eDiscovery & Compliance, Enterprise AI, Enterprise eDiscovery, Enterprise Search, ESI, Information Governance, Information Management

Bringing AI to the Data: How X1 Search v11 Redefines Secure Enterprise Search

By John Patzakis

At X1, we believe the future of enterprise AI depends on a simple but often overlooked principle: data should not have to move in order to become intelligent. With the launch of X1 Search v11, we are introducing a fundamentally different approach—one that embeds AI directly into our index-in-place architecture. Rather than forcing organizations to centralize and copy their data into external platforms, we enable AI to operate exactly where that data already lives. You can read the full press release here: https://www.x1.com/x1-introduces-ai-powered-x1-search-delivering-secure-ai-in-place-for-individual-and-enterprise-users/

This release represents an important milestone for us and for our customers. As Chas Meier noted, “X1 Search v11 marks an important milestone in how organizations can safely apply AI…without compromising the security controls enterprise environments demand.” That statement reflects our core design philosophy: AI must adapt to enterprise security, compliance, and governance requirements—not the other way around.

With X1 Search v11, we are delivering AI capabilities directly within our micro-index. That means organizations can apply advanced intelligence—classification, categorization, and contextual analysis—across emails, files, and collaboration data without ever relocating that information. Everything happens in place, within existing security boundaries, whether on endpoints or across enterprise systems.

For large enterprises, this architecture unlocks an even more powerful capability: the ability to deploy their own trained and curated large language models directly into the X1 index. Instead of relying solely on generic, hosted AI services, organizations can operationalize models tailored to their data that reflect their internal policies, regulatory requirements, and business workflows. These models run directly against their data, in place, delivering highly relevant and controlled outcomes.

This approach stands in sharp contrast to traditional hosted AI platforms. In those models, organizations must copy and transfer massive amounts of sensitive data into third-party hosted AI platforms before any meaningful analysis can occur. That process introduces serious risks. Moving data to outside providers complicates compliance, potentially compromises IP, and creates new attack surfaces that most enterprises simply cannot accept.

Beyond security concerns, the traditional model also breaks down operationally at scale. Enterprises are not dealing with small data sets; they are managing dozens of terabytes of distributed, unstructured data. Attempting to duplicate and transfer that volume is not just costly; it is infeasible. The result is delays, fragmentation, and incomplete analysis—undermining the very promise of AI.

We have taken a different path. By bringing AI to the data through our distributed micro-indexing technology, we eliminate the need for data movement entirely. Models can be deployed directly to where data resides, enabling real-time analysis while preserving security, reducing infrastructure overhead, and scaling seamlessly across the enterprise.

We see X1 Search v11 as more than a product release—it is a shift in how enterprise AI is deployed. Organizations no longer have to choose between innovation and control. With AI in place, they can achieve both.

To see this in action, we invite you to join our upcoming live product tour on Thursday, April 23, providing a guided walkthrough of the new AI-enriched capabilities and flexible model deployment features.

Leave a comment

Filed under Best Practices, Business Productivity Search, Desktop Search, Enterprise AI, Enterprise eDiscovery, Enterprise Search, ESI, Google Workspace, Information Access, Information Management, m365, MS Teams, X1 Search 11

Modernizing eDiscovery: A Huge Strategic Win for Legal Operations Executives

By John Patzakis

Modern In-Place Data Discovery

For today’s corporate legal departments, controlling runaway costs is no longer optional — it’s a mandate. Nowhere is this more evident than in the spiraling expenses for outsourced eDiscovery and information governance services. While litigation and regulatory demands continue to grow, many organizations still rely heavily on costly outside service providers to identify, collect, process, and produce electronically stored information (ESI). This outdated model drains budgets, strains timelines, and introduces unnecessary risk.

Enter the modern legal operations executive. One of their core responsibilities is to identify inefficiencies and leverage technology to reduce costs and streamline workflows. Modernizing eDiscovery and information governance processes is a very fertile and high-impact opportunity to do exactly that. Doing so can save organizations tens of millions of dollars in hard (actual) costs. Here’s how:

1) Bring eDiscovery In-House and Slash Costs with the Right Technology

Outsourced eDiscovery vendors typically charge steep hourly rates and volume-based markups for even routine tasks like identifying and collecting custodial data. Yet studies — and real-world case studies — consistently show that corporations can reduce eDiscovery costs by up to 90% by adopting targeted collection and in-place search technology.

Solutions like X1 Enterprise enable legal and compliance teams to index and search data in place — without cumbersome, time-consuming manual collection. By deploying this technology internally, the legal operations team can replace costly third-party workflows, including highly inefficient Microsoft 365 processes, with faster, defensible, and far less expensive processes. This means greater control over timelines and budgets, and reduced exposure to data security risks associated with handing over large volumes of sensitive information to multiple vendors.

2) Drive Broader Efficiencies Beyond Litigation

The benefits of a modern eDiscovery platform extend far beyond document production in a lawsuit. The same technology can be leveraged for critical information governance and data compliance functions. For example, when a company needs to respond to internal audits, regulatory data access requests, or data privacy audits and inquiries, in-place search capabilities allow teams to quickly find and manage relevant data without reinventing the wheel each time.

Legal operations executives can champion the use of enterprise eDiscovery tools for these broader use cases, creating synergies between compliance, privacy, IT, and legal teams. This not only reduces redundant spending on separate point solutions but also ensures better control of data and improved risk management across the organization.

3) Partner with Finance to Uncover Hidden Cost Savings

A key role of legal operations is to align legal spend with broader corporate financial goals. When evaluating an in-house eDiscovery solution, legal ops leaders should engage their CFO early. One common pitfall is focusing solely on capital IT budgets while overlooking how much is siphoned away from the legal operating budget to fund expensive outsourced eDiscovery services.

In one real-world example, a company assumed they could not afford an internal solution based on their limited IT budget. However, when they worked with their CFO to analyze total eDiscovery spending, they discovered they were paying tens of millions annually from a separate operating budget to outside providers. Redirecting even a fraction of this spend towards a robust internal platform not only paid for the technology but will yield millions in net savings — year after year.

Final Thoughts

For legal operations executives looking to deliver immediate cost savings, increase efficiency, and elevate the department’s strategic value, modernizing eDiscovery and information governance processes is perhaps their greatest opportunity for an immediate and significant impact. By bringing the process in-house with proven technology like X1 Enterprise, expanding its use to multiple compliance and governance scenarios, and partnering with finance to eliminate wasteful spending, legal operations can transform eDiscovery and information governance from a financial drain into a model of operational excellence.

Interested in learning more about how to achieve this transformation? Schedule a briefing today at sales@x1.com or visit www.x1.com/solutions/x1-enterprise-platform.

Leave a comment

Filed under Best Practices, Cloud Data, Corporations, Data Audit, ECA, eDiscovery, eDiscovery & Compliance, Enterprise eDiscovery, Enterprise Search, ESI, Information Access, Information Governance, Information Management, m365, Preservation & Collection, Records Management

X1 Enterprise Is the Gold Standard for Data Separation in M&A Matters

By John Patzakis and Charles Meier

X1 is the Gold Standard in Data Separation

Corporate mergers and acquisitions are complex enough on their own — but when a deal involves the divestiture of an entire business unit or a carve-out of specific departments, the stakes for separating data correctly and efficiently become even higher. Legal and IT teams must identify and surgically separate emails, documents, and other unstructured electronic information to ensure that the right data goes to the acquiring party — and that what must be retained remains secure and compliant with privacy and legal requirements.

This data separation exercise is notorious for being time-consuming, extremely expensive, and highly disruptive. This is because traditional methods require heavy lifting by IT teams and service providers, endless back-and-forth with custodians, and mass data collections that literally double the risk. Worse yet, Microsoft Purview, with its known throttling and low throughput challenges for M 365 data, is not up to the task for data separation matters that invariably involve at least dozens of terabytes. These inefficiencies all lead to severe regulatory risks, runaway costs, and critical delays.

There is, however, a far better way — X1 Enterprise. Several major corporations have recently employed X1 Enterprise in high-stakes data separation matters. Once completed, the comments from our customers are the same: There was no other way they could have done it without spending millions of dollars on time-consuming and disruptive services.

Data Separation Is Not Just Another eDiscovery Project

Unlike standard eDiscovery, a divestiture-driven data separation project must carve out large volumes of live, operational data while the business continues to run. Legacy tools and processes require copying and moving the entire subject data set to a separate repository for indexing and searching — adding huge costs, time delays, and operational risk.

X1 Enterprise’s game-changing advantage lies in its distributed micro-indexing architecture and true index-in-place capability. This unique approach allows organizations to instantly search, categorize, and separate or otherwise remediate massive volumes of data where it resides — without duplicating and exporting entire data sets to third-party servers for processing.

In practical terms, this means:

Lightning-Fast Search: X1 Enterprise creates lightweight, local micro-indexes on endpoints and servers across the organization. Search results come back in seconds, no matter where the data lives — on laptops, file shares, or cloud repositories such as M365.

Minimal Disruption: Because the data stays in place, there is no need to duplicate or move sensitive content, minimizing the risk of data leakage, avoiding the bottlenecks that come with data copying and migration for centralized processing, and enabling the actual remediation to be infinitely more effective by working on the live data set. How do you execute data separation when you are working off a stale copy of the data for the categorization effort? The short answer: Up to millions of dollars in manual services to go back to the “original data” and manually separate the data for each employee and their respective data sources.

Scalability and Control: Whether the divestiture involves hundreds or thousands of custodians across geographies, X1 Enterprise scales seamlessly while giving legal and IT teams centralized control and real-time oversight.

Defensible Process: Legal teams can generate audit trails, reports, and logs to demonstrate a precise and defensible chain of custody, which is critical for regulatory and contractual compliance.

The Bottom Line: Much Faster, with Dramatically less Cost and Risk.

When time is money — and delays can put entire deals at risk — organizations cannot afford cumbersome, legacy eDiscovery workflows for carve-out data separation projects. X1 Enterprise’s innovative architecture empowers legal, compliance, and IT teams to execute precise data separations faster, with dramatically lower cost and business impact.

For any organization facing a merger, acquisition, or divestiture, X1 Enterprise is not just an upgrade — it is the modern standard for high-stakes data separation and governance.

Learn more about how X1 Enterprise can streamline your next M&A project. Schedule a demo today at sales@x1.com or visit  www.x1.com/solutions/x1-enterprise-platform.

Leave a comment

Filed under Best Practices, Case Study, Cloud Data, compliance, Corporations, Data Audit, ECA, eDiscovery & Compliance, Enterprise eDiscovery, ESI, GDPR, Information Access, Information Governance, Information Management, m365, Preservation & Collection, Records Management