Category Archives: Information Governance

True Enterprise-Wide eDiscovery Collection is Finally Here

My previous post discussed the inability of any software provider to solve a critical need by delivering a truly scalable eDiscovery preservation and collection solution that can search across thousands of enterprise endpoints in a short period of time. In the absence of such a “holy grail” solution, eDiscovery collection remains dominated by either unsupervised custodian self-collection or manual services, driving up costs while increasing risk and disruption to business operations.

So today, we at X1 are excited to announce the release of X1 Distributed Discovery. X1 Distributed Discovery (X1DD) enables enterprises to quickly and easily search across up to tens of thousands of distributed endpoints and data servers from a central location.  Legal and compliance teams can easily perform unified complex searches across both unstructured content and metadata, obtaining statistical insight into the data in minutes, and full results with completed collection in hours, instead of days or weeks. Built on our award-winning and patented X1 Search technology, X1DD is the first product to offer true and massively scalable distributed data discovery across an organization. X1DD replaces expensive, cumbersome and highly disruptive approaches to meet enterprise discovery, preservation, and collection needs.


Enterprise eDiscovery collection remains a significant pain point, subjecting organizations to both substantial cost and risk. X1DD addresses this challenge by starting to show results from distributed data across global enterprises within minutes instead of today’s standard of weeks, and even months. This game-changing capability vastly reduces costs while greatly mitigating risk and disruption to operations.

Targeted and iterative end point search is a quantum leap in early data assessment, which is critical to legal counsel at the outset of any legal matter. However, under today’s industry standard, the legal team is typically kept in the dark for weeks, if not months, as the manual identification and collection process of distributed, unstructured data runs its expensive and inefficient course.  To illustrate the power and capabilities of X1DD, imagine being able to perform multiple detailed Boolean keyword phrase searches with metadata filters across the targeted end points of your global enterprise. The results start returning in minutes, with granular statistical data about the responsive documents and emails associated with specific custodians or groups of custodians.

Once the legal team is satisfied with a specific search string, after sufficient iteration, the data can then be collected by X1DD by simply hitting the “collect” button. The responsive data is “containerized” at each end point and automatically transmitted to a central location, where all data is seamlessly indexed and ready for further culling and first pass review. Importantly, all results are tied back to a specific custodian, with full chain of custody and preservation of all file metadata.

This effort described above — from iterative distributed search through collection, transmittal to a central location, and indexing of data from thousands of endpoints — can be accomplished in a single day. Using manual consulting services, the same project would require several weeks and hundreds of thousands of dollars in collection costs alone, not to mention significant disruption to business operations. Substantial costs associated with over-collection of data would mount as well.

X1DD operates on-demand where your data currently resides — on desktops, laptops, servers, or even the Cloud — without disruption to business operations and without requiring extensive or complex hardware configurations. Beyond enterprise eDiscovery and investigation functionality, organizations can offer employees the award-winning X1 Search, improving productivity while maintaining compliance.

X1DD will be featured in an April 19 webinar with eDiscovery expert Erik Laykin of Duff & Phelps. Watch a full briefing and technical demo of X1DD and find out for yourself why X1 Distributed Discovery is a game-changing solution. Or please contact us to arrange for a private demo.

Leave a comment

Filed under Best Practices, Corporations, Desktop Search, eDiscovery, eDiscovery & Compliance, Enterprise eDiscovery, Information Governance, Information Management, Preservation & Collection, X1 Search 8

Enterprise eDiscovery Collection Remains Costly and Inefficient

2016 marks my sixteenth year as a senior executive in the eDiscovery business. I began my career as a co-founder at Guidance Software (EnCase), serving as General Counsel, CEO and then Vice Chairman and Chief Strategy Officer from 1999 through 2009. After becoming the dominant solution for computer forensics in the early part of the last decade, Guidance set out to define a new field — enterprise discovery collection. Despite a good foundational concept, a truly scalable solution that could search across hundreds, or even thousands, of enterprise endpoints in a short period of time never came to fruition. To date, no other eDiscovery vendor has delivered on the promise of such a “holy grail” solution either. As a result, eDiscovery collection remains dominated by either unsupervised custodian self-collection, or manual services.



Organizations employ limited technical approaches in an effort to get by, and thus enterprise eDiscovery collection remains a significant pain point, subjecting organizations to both significant cost and risks. This post is the first of a two part series on the status of the broken enterprise eDiscovery collection process. Part two will outline a proposed solution.

Currently, enterprises employ four general approaches to eDiscovery collection, with two involving mostly manual methodologies, and the other two predominantly technology-based. Each of the four methods are fraught with inefficiencies and challenges.

The first and likely most common approach, is custodian self-collection, where custodians are sent manual instructions to search, review and upload data that they subjectively determine to be responsive to a matter. This method is plagued with severe defensibility concerns, with several courts disapproving of the practice due to poor compliance, modifying metadata, and inconsistency of results. See Geen v. Blitz, 2011 WL 806011, (E.D. Tex. Mar. 1, 2011), Nat’l Day Laborer Org. v. U.S. Immigration and Customs Enforcement Agency, 2012 WL 2878130 (S.D.N.Y. July 13, 2012).

The second approach is manual services, usually performed by eDiscovery consultants. This method is expensive, disruptive and time-consuming as many times an “overkill” method of forensic image collection process is employed. It also often results in over collection, as the collector typically only gets one bite at the apple, thus driving up eDiscovery costs. While attorney review and processing represent the bulk of eDiscovery costs, much of these expenses stem from over-collection, and thus can be mitigated with a smarter and more efficient process.

When it comes to technical approaches, endpoint forensic crawling methods are employed on a limited basis. While this can be feasible for a small number of custodians, network bandwidth constraints coupled with the requirement to migrate all endpoint data back to the forensic crawling tool renders the approach ineffective. For example, to search a custodian’s laptop with 10 gigabytes of email and documents, all 10 gigabytes must be copied and transmitted over the network, where it is then searched, all of which takes at least several hours per computer. So, most organizations choose to force collect all 10 gigabytes. The case of U.S. ex rel. McBride v. Halliburton Co.  272 F.R.D. 235 (2011), Illustrates this specific pain point well. In McBride, Magistrate Judge John Facciola’s instructive opinion outlines Halliburton’s eDiscovery struggles to collect and process data from remote locations:

“Since the defendants employ persons overseas, this data collection may have to be shipped to the United States, or sent by network connections with finite capacity, which may require several days just to copy and transmit the data from a single custodian . . . (Halliburton) estimates that each custodian averages 15–20 gigabytes of data, and collection can take two to ten days per custodian. The data must then be processed to be rendered searchable by the review tool being used, a process that can overwhelm the computer’s capacity and require that the data be processed by batch, as opposed to all at once.”

Halliburton represented to the court that they spent hundreds of thousands of dollars on eDiscovery for only a few dozen remotely located custodians. The need to force-collect the remote custodians’ entire set of data and then sort it out through the expensive eDiscovery processing phase, instead of culling, filtering and searching the data at the point of collection drove up the costs.

And finally, another tactic attempted by some CIOs to attempt to address this daunting challenge is to periodically migrate disparate data from around the global enterprise into a central location. This Quixotic endeavor is perceived necessary as traditional information management and electronic discovery tools are not architected and not suited to address large and disparate volumes of data located in hundreds of offices and work sites across the globe.  But, boiling the ocean through data migration and centralization is extremely expensive, disruptive and frankly unworkable.

What has always been needed is gaining immediate visibility into unstructured distributed data across the enterprise, through the ability to search and collect across several hundred endpoints and other unstructured data sources such as file shares and SharePoint, and return results within minutes instead of days or weeks. None of the four approaches outlined above come close to meeting this requirement and in fact actually perpetuate eDiscovery pain.

Is there a fifth option? Stay tuned for my next post coming soon.

Leave a comment

Filed under Best Practices, Case Law, eDiscovery, eDiscovery & Compliance, Enterprise eDiscovery, Information Governance, Information Management, Preservation & Collection

Using Search to Solve the Email Overload Problem

by Barry Murphy

There was an interesting article from The Information Governance Initiative about email overload and information governance. The quote that caught my eye is “in today’s fast-paced business world, the name of the game is productivity.”  X1 Search is a tool that many are using for just that – business productivity search. But, more than that, X1 Search can complement information governance efforts and help solve the email overload problem.

The IGI article states that “IG practitioners need to take a proactive approach in order to truly understand the realities of email overload and the entire scope of their organizations’ communications.”  I believe that IG is extremely important, especially in an information economy. The challenge is that it can be hard for IG to gain real traction unless companies have a senior executive with clout, power, and money who cares about it. Whether or not that executive exists at a given organization is hit or miss at best. IG programs will take time to gain traction and help employees be more productive.

But the email overload problem continues to exist. And, it is not simply about inbox management. Rather, it is across email that is on a server, or in an archive, or in a PST file. It is easy to forget that many organizations still have PST files on desktops or have moved them out to file servers. There is a customer using X1 to solve the “PSTs on file shares” problem.  What this customer does is use X1 Rapid Discovery to index the PSTs once, and then use X1 Search as the interface to that information. Users can quickly filter through years of emails to find exactly what they are looking for and to take action on it.

This customer uses X1 to complement IG processes and policies. With Rapid Discovery, those PSTs are now easily discoverable for litigation needs, instead of the previous need to forensically image desktops to get at the information. This creates a win-win and negates the need to do expensive migration of PSTs to an archive or to the cloud. At the same time, employees can deal with the email overload problem better because they are much faster at finding the right emails to do their jobs. It is an interesting use-case whereby the customer solved several problems at once and did so in a pragmatic way. The employee happiness with the X1 Search tool is the cherry on top of the sundae because it is a lasting benefit over the long-term.

IG projects can be painful and time-consuming and, if funded properly, often go nowhere. Fitting X1 into your IG program can save time, save money, and keep employees productive at the same time. For any organization seeking a quick IG win to prove the value IG brings to the company, X1 should be on the list.

Leave a comment

Filed under Best Practices, Business Productivity Search, Enterprise Search, Information Governance

Amazon Re:Invent – With the Cloud, Avoid Mistakes of the Past

Last week, I had the opportunity to attend the Amazon Re:Invent conference in Las Vegas. Over 13,000 people took over the Palazzo for deep dive technical sessions to learn how to harness the power of Amazon Web Services (AWS). reinventThis show had a much different energy than other enterprise software conferences, such as VMworld.  Whereas most conferences feature a great deal of selling and marketing by the host, Amazon Re:Invent was truly more of a training show. Cloud architects spent a lot of time in technical bootcamps learning how AWS works and getting certified as administrators.

That is not to say that there was no selling or marketing going on; the exhibition hall featured myriad vendors that augment or assist with AWS deployments and solutions. The focus on the deep technical details, though, does point out the fact that we are still in the very early days of the cloud. Most of the focus of the keynotes was about getting compute workloads to the cloud – there was not a lot of mention of moving actual data to the cloud, even though that is certainly beginning to happen.  But, that is how the evolution goes. IT departments need to be comfortable moving workloads to the cloud as they begin to leverage the cloud. Building this foundation is also important to Amazon – the goal would be for many companies to completely outsource the IT data center.

It is important, however, to proactive plan for information management as more workloads and, importantly, data move to the cloud.  As the internet first emerged, companies dove into new technologies like email and network file shares only to create eDiscovery nightmares and make it virtually impossible to find information within digital landfills. It is key to learn from those mistakes rather than to repeat them when leveraging cloud-based technologies. In order to ensure both that end-users are happy with search experiences on data in the cloud and that Legal can do what they need to do from an eDiscovery standpoint. This means providing business workers with unified access to email, files, and SharePoint information regardless of where the data lives. It also means giving Legal teams fast search queries and collections. But, Cloud search is slow, as indexes live far from the information. This results in frustrated workers and Legal teams afraid that eDiscovery cannot be completed in time.

If a customer wanted to speed up search, it would have to essentially attach an appliance to a hot-air balloon and send it up to the Cloud provider so that the customer’s index could live on that appliance (or farm of appliances) in the Cloud providers data center, physically near the data. There are many reasons, however, that a Cloud provider would not allow a customer to do that:

  • Long install process
  • Challenging Pre-requisites
  • 3rd party installation concerns
  • Physical access
  • Specific hardware requirements
  • They only scale vertically

The solution to a faster search is a cloud-deployable search application, such as X1 Rapid Discovery. This creates a win-win for Cloud providers and customers alike. As enterprises move more and more information to the Cloud, it will be important to think about workers’ experiences with Cloud systems – and search is one of those user experiences that, if it is a bad one, can really negatively affect a project and cause user revolt. eDiscovery is also a major concern – I’ve worked with organizations that moved data to the cloud before planning how they would handle eDiscovery. That left Legal teams to clean up messes, or more realistically, just deal with the messes. By thinking about these issues before moving data to the cloud, it is possible to avoid these painful occurrences and leverage the cloud without headaches. At X1, we look forward to working closely with Amazon to help customers have the search and eDiscovery solutions they need as more and more data goes to AWS.

Leave a comment

Filed under Cloud Data, eDiscovery & Compliance, Enterprise eDiscovery, Enterprise Search, Hybrid Search, Information Access, Information Governance, Information Management

Seeing The Full Picture On Hybrid Cloud

If it seems that a lot has been written about hybrid cloud lately, that’s because there has – it is one of the hottest topics in the technology world, if not the hottest.  hybrid cloudThe hybrid cloud is a combination of a private IT infrastructure and a public cloud.  The public and private cloud infrastructures then communicate over an encrypted connection and can port data and applications back and forth.  Hybrid cloud is hot because it delivers real benefits:  increased speed of access time and reduced latency because of an on-premise, private infrastructure that is accessible directly as opposed to through the internet; more flexibility to have on-premises infrastructure that can support the average workload and to leverage the public cloud when the workload exceeds the power of the private cloud component; and more flexibility in server designs that can lower the costs of storage.

These benefits (there are many more, but the list would be too long) have IT departments excited to leverage hybrid cloud.  As organizations gain experience with hybrid cloud, we are seeing more and more written about it.  Most of what is written focuses on the hard-core IT issues.  Industry blogs often dig deep in the ability to port applications from on-premise to the cloud and back without requiring re-architecting the apps or hitting major bumps in the workload function.  Or, they might be about the ability to migrate server workloads to the cloud.  This is clearly important stuff, but it is only painting half the picture.   No one is talking much about where the information feeding these applications lives, or about how to ensure the information is accessible as needed.

This is why we need to see the full picture on hybrid cloud.  The reality is the information will live all over the place and business workers will need unified access to it, without having to know the location.   We should be talking about hybrid search equally as much as we talk about the other issues related to hybrid cloud. This is because end-user search experience is extremely important to executing successful IT projects.  We have seen this up-close-and-in-person in the VDI market.  Many organizations rolled out virtual desktops to employees and followed the best practice of turning off Windows indexing.  When users went to search for their information, they were unable to do so and revolted.  That is a lose-lose scenario.  The solution, in that case, is X1 Search Virtual Edition – the only search solution that is architected specifically for VDI environments.

The lesson from VDI is simple:  do not forget the business workers that will need to do their jobs (which tends to require finding their important emails and files quickly and efficiently).   Products like X1 Rapid Discovery enable hybrid search that lets IT glean all the benefits of hybrid cloud while ensuring end-users are happy with their ability to find information.  If we learn from that lesson as we venture into the hybrid cloud, we can avoid the nightmares that come when users are less than thrilled with the solutions IT rolls out to them.  If we think about hybrid search now, IT departments embracing hybrid cloud can be heroes to the C-Level executives tracking performance and to the business workers they serve.


Leave a comment

Filed under Cloud Data, Hybrid Search, Information Access, Information Governance, Information Management, Virtualized Environment