Tag Archives: cloud

Federal Government Agencies Face Information Management Challenges, Too

by Barry Murphy

Many moons ago, one of my first projects as an analyst with Forrester Research was to find the answer to a seemingly simple question: what is the industry standard for storing new types of electronic information such as X-rays and other images?  The client was a government agency that needed to store these records long-term and anticipated potentially needing to produce them in court many years in the future.  fed image 2As such, the agency needed to know how to store and find these records.  The answer proved to be anything but simple – in reality, the answer was that there was no “standard” for storing this new type of content.  My investigation into the topic led me to find this new market called eDiscovery and the rest is history.

The experience was interesting because it was my first foray into working with the Federal government.  I went into the experience with the assumption that Federal agencies would somehow be more advanced in their information management efforts.  Records management, after all, was (and still is) very important in government.  But, government agencies are just like other organizations – struggling to keep up with exploding volumes of digital information, under the gun to respond to information requests (whether it is in response to Congressional inquiries, regulatory requests, or litigation), and dealing with the needs for more modern IT infrastructure elements like Virtual Desktop Infrastructure (VDI).

The only difference between government agencies and other organizations is that agencies may be under even more stringent rules for complying with investigations and for dealing with digital information management.  President Obama’s Managing Government Records directive mandates that agencies manage electronic information as stringently as paper records have traditionally been managed.  Agencies are under even more of a microscope than corporations, yet face the same information management challenges.  Information assets are scattered across email, file systems, disparate SharePoint sites, and Cloud-based repositories.  In addition, some agencies adopt newer IT infrastructure elements such as virtualization and cloud computing to stay relevant.  For example, the Department of Energy deployed a virtual desktop infrastructure (VDI) in order to enable worker mobility (Source: Zurier, Steve. Agencies Deploy VDI with an Eye Toward BYOD. Fed Tech Magazine, March 18, 2013). VDI allows agencies to implement initiatives like BYOD while maintaining top-notch security.

Keeping up with modern IT infrastructure, while simultaneously responding to thousands of information requests each year – whether it is in response to Congressional inquiries, regulatory requests, or litigation – is a challenge.  Before assuming that government agencies have the process under control, consider this: according to Deloitte’s Seventh Annual Benchmarking Study of Electronic Discovery Practices for Government Agencies, only 59% of respondents believed their agencies were effective in deploying eDiscovery capabilities compared to 73% in the previous year (Source: Deloitte’s Seventh Annual Benchmarking Study of Electronic Discovery Practices for Government Agencies, Spring 2013).

Why the drop in confidence?  Part of the reason is that traditional search and eDiscovery products fail to effectively support agencies’ IT environments in a way that creates a true solution.  Rather, traditional products have agencies creating centralized eDiscovery labs that require copying information from various systems to a central eDiscovery location.  This is both time-consuming and expensive.  To learn how to address information management challenges in federal agencies, click here to download a whitepaper that outlines the critical problem, its legal compliance implications, and compelling solutions that help agencies develop built-in search and eDiscovery capabilities that reduce costs and improve operational productivity.

1 Comment

Filed under Information Management, Records Management

Amazon to Host Cloud Webinar on eDiscovery and Enterprise Search

Amazon Web Services2On June 27, Amazon Web Services (AWS) will be hosting a first of its kind webinar by a major cloud provider addressing the topics of eDiscovery and enterprise search.  Per AWS, the webinar will explore solutions that allow organizations to quickly search, identify and act upon distributed data, whether it resides within the enterprise or within the AWS cloud. Vikram Garlapati, an Amazon Web Service Solutions Architect, will lead off the discussion. This is the first time, to our knowledge, that Amazon will be publically addressing eDiscovery in the cloud and featuring a solution like X1 and also the first time they will be featuring a third party enterprise search solution to enable search within the IaaS cloud.

While SaaS hosting of eDiscovery services and some limited enterprise search capabilities by a vendor from their own servers is not new, this webinar will effectively be addressing a very different topic: deploying search solutions for eDiscovery and business productivity into an organizations’ public cloud instance where their data resides. This allows instant and lightning fast indexing, searching, and review of that data in place without having to first export that data out of its native cloud environment.

Also presenting will be LTech CIO Eric Klotzko.  LTech is a cloud systems integrator and AWS partner who is a major proponent of next-generation enterprise search and eDiscovery solutions that install and operate in virtual environments.  Eric will underscore the limitations of traditional enterprise search solutions that are hardware appliance-based or require an extensive manual on-site install process, thereby rendering such solutions as non-starters for deploying into and operating within virtualized cloud deployments.

This is a compelling, very timely topic that impacts the overall enablement and adoption of the cloud. As I blogged about previously, Federal Court Magistrate Judge Andrew Peck of the New York Southern District recently weighed in on public comments on eDiscovery in the cloud. Judge Peck noted that data stored in the cloud is considered accessible data under the Federal Rules of Civil Procedure (see, FRCP Rule 26(b)(2)(B)) and thus treated no differently by the courts in terms of eDiscovery preservation and production requirements as data stored within a traditional network. This brought the following cautionary tale about the costs associated with not having a systematic process for eDiscovery:

Judge Peck told the story of a Chief Information Security Officer who had authority over e-discovery within his multi-billion dollar company who, when told that the company could enjoy significant savings by moving to “the cloud”, questioned whether the cloud provider could accommodate their needs to adapt cloud storage with the organization’s e-discovery preservation requirements. The cloud provider  said it could but at such an increased cost that the company would enjoy no savings at all if it migrated to the cloud.

Having just visited Amazon Web Services headquarters office and meeting with several key senior managers, it is clear this issue is a priority for AWS. This is precisely the reason why we developed X1 Rapid Discovery, version 4. X1RD is a proven and now truly cloud-deployable eDiscovery and enterprise search solution enabling our customers to quickly identify, search, and collect distributed data wherever it resides in the Infrastructure as a Service (IaaS) cloud or within the enterprise.

As far as the major cloud providers, the ones who choose to solve this eDiscovery challenge (along with effective enterprise search) with best practices technology will not only drive significant managed services revenue but will enable a far more rapid adoption of cloud computing.

Please stay tuned for more exciting developments in this next frontier of eDiscovery and enterprise search.

A recording of the webinar is available here >

Leave a comment

Filed under eDiscovery & Compliance, Enterprise eDiscovery, IaaS, Virtualized Environment

Judge Peck: Cloud For Enterprises Not Cost-Effective Without Efficient eDiscovery Process

Hon. Andrew J. Peck
United States Magistrate Judge

Federal Court Magistrate Judge Andrew Peck of the New York Southern District is known for several important decisions affecting the eDiscovery field including the ongoing  Monique da Silva Moore v. Publicis Group SA, et al, case where he issued a landmark order authorizing the use of predictive coding, otherwise known as technology assisted review. His Da Silva Moore ruling is clearly an important development, but also very noteworthy are Judge Peck’s recent public comments on eDiscovery in the cloud.

eDiscovery attorney Patrick Burke, a friend and former colleague at Guidance Software, reports on his blog some interesting comments asserted on the May 22 Judges panel session at the 2012 CEIC conference. UK eDiscovery expert Chris Dale also blogged about the session, where Judge Peck noted that data stored in the cloud is considered accessible data under the Federal Rules of Civil Procedure (see, FRCP Rule 26(b)(2)(B)) and thus treated no differently by the courts in terms of eDiscovery preservation and production requirements as data stored within a traditional network. This brought the following cautionary tale about the costs associated with not having a systematic process for eDiscovery:

Judge Peck told the story of a Chief Information Security Officer who had authority over e-discovery within his multi-billion dollar company who, when told that the company could enjoy significant savings by moving to “the cloud”, questioned whether the cloud provider could accommodate their needs to adapt cloud storage with the organization’s e-discovery preservation requirements. The cloud provider said it could but at such an increased cost that the company would enjoy no savings at all if it migrated to the cloud.

In previous posts on this blog, we outlined how significant cost-benefits associated with cloud migration can be negated when eDiscovery search and retrieval of that data is required.  If an organization maintains two terabytes of documents in the Amazon or other IaaS cloud deployments, how do they quickly access, search, triage and collect that data in its existing cloud environment if a critical eDiscovery or compliance search requirement suddenly arises?  This is precisely the reason why we developed X1 Rapid Discovery, version 4. X1RD is a proven and now truly cloud-deployable eDiscovery and enterprise search solution enabling our customers to quickly identify, search, and collect distributed data wherever it resides in the Infrastructure as a Service (IaaS) cloud or within the enterprise. While it is now trendy for eDiscovery software providers to re-brand their software as cloud solutions, X1RD is now uniquely deployable anywhere, anytime in the IaaS cloud within minutes. X1RD also features the ability to leverage the parallel processing power of the cloud to scale up and scale down as needed. In fact, X1RD is the first pure eDiscovery solution (not including a hosted email archive tool) to meet the technical requirements and be accepted into the Amazon AWS ISV program.

As far as the major cloud providers, the ones who choose to solve this eDiscovery challenge (along with effective enterprise search) with best practices technology will not only drive significant managed services revenue but will enjoy a substantial competitive advantage over other cloud services providers.

Leave a comment

Filed under Best Practices, Case Law, Cloud Data, Enterprise eDiscovery, IaaS, Preservation & Collection

Defining Truly Cloud-Capable eDiscovery Software

Last week we discussed the challenges of searching and collecting data in Infrastructure as a Service (IaaS) cloud deployments (such as the Amazon cloud or Rackspace) for eDiscovery purposes.  Today we discuss what is needed for eDiscovery and enterprise search vendors to provide a truly cloud-capable solution and provide a decoder ring of sorts to cut through the hype.  For there is a lot of hype with the cloud becoming the latest eDiscovery hot button, with vendor marketing claims far surpassing actual capabilities.

In fact, many eDiscovery and enterprise software vendors claim to support the cloud, but are simply re-branding their long-existing SaaS offerings, which really has nothing to do with supporting IaaS. Barry Murphy of the eDiscovery Journal aptly identified this marketing practice as “cloud washing.” Data hosting, especially where the vendor’s manual labor is routinely required to upload and process data, does not meet defined cloud standards. Neither does a process that primarily exports data through APIs or other means out of its resident cloud environment to slowly migrate the cloud data to the vendor tools, instead of deploying the tools (and their processing power) to the data where it resides in the cloud. In order to truly support IaaS cloud deployments, eDiscovery and enterprise search software must meet the following three core requirements:

1.         Automated installation and virtualization:  The eDiscovery and search solution must immediately and rapidly install, execute and efficiently operate in a virtualized environment without rigid hardware requirements or on-site physical access. This is impossible if the solution is fused to hardware appliances or otherwise requires a complex on-site installation process. As hardware appliance solutions by definition are not cloud deployable and with enterprise search installations often requiring many months of man hours to install and configure, whether many of these vendors will be able to support robust IaaS cloud deployments in the reasonably foreseeable future is a significant question.

2.         On-demand self-service: In its definition of cloud computing, The National Institute of Standards and Technology (NIST) identifies on- demand self-service as an essential characteristic of the cloud where a “consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider.”

Many hosted eDiscovery services require shipping of data to the provider or extensive behind the scenes manual labor to load and configure the systems for data ingestion. Conversely, solutions that truly support cloud IaaS will spin up, ingest data and fully operate in an automated fashion without the need for manual on-premise labor for configuration or data import.

3.         Rapid elasticity: NIST describes this characteristic as capabilities that “scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time.” This important benefit of cloud computing is accomplished by a parallelized software architecture designed to dynamically scale out over potentially several dozen virtualized servers to enable rapid ingestion, processing and analysis of data sets in that cloud environment. This capability would allow several terabytes of data to be indexed and processed within 2 to 4 hours on a highly automated basis at far less cost than non-cloud eDiscovery efforts.

However, many characteristics of leading eDiscovery solutions fundamentality prevent their ability to support this core cloud requirement. Most eDiscovery early case assessment solutions are developed and configured toward a monolithic processing schema designed to operate on a single expensive hardware apparatus. While recently spawning some bold marketing claims of high speeds and feeds, such architecture is very ill-suited to the cloud, which is powered by highly distributed processing across multitudes of servers. Additionally, many of the leading eDiscovery and enterprise search solutions are tightly integrated with third party databases and other OEM technology that cannot be easily decoupled (and also present possible licensing constraints) making such elasticity physically and even legally impossible.

So is there eDiscovery software that will truly support the IaaS cloud based upon these requirements, and address up to terabytes of data?  Stay tuned….

Leave a comment

Filed under Cloud Data, Enterprise eDiscovery, IaaS

eDiscovery Search and Collection in the Cloud

After several dozen posts on social media eDiscovery, we are going to focus the next few weeks on the related issue of eDiscovery in the cloud. As we see it, despite the enormous cost benefits of the cloud, concerns about the feasibility of eDiscovery and general search across an organization’s critical cloud-resident data has to some degree prevented broader adoption.

The cloud means many things to many people, but I believe the real eDiscovery action (and pain point) is in Infrastructure as a Service (IaaS) cloud deployments (such as the Amazon cloud, Rackspace, or pure enterprise cloud providers such as Fujitsu). According to a recent PwC report, Cloud IaaS will account for 30% of IT expenditures by 2014.  IaaS currently provides the means for organizations to aggressively store and virtualize their enterprise data and software, thus potentially spawning the same large data volumes and requiring the same critical search and eDiscovery requirements as traditional enterprise environments.  Amazon Web Services, the leading IaaS cloud provider, reports in our discussions with them extensive customer eDiscovery requirements that are currently addressed by inefficient and manual means.  So for purposes of this discussion, IaaS, which is essentially cloud for the enterprise and where there is a current significant eDiscovery challenge, is what we will focus on.

So if an organization maintains two terabytes of documents in the Amazon or Rackspace cloud, how do they quickly access, search, triage and collect that data in its existing cloud environment if a critical eDiscovery or compliance search requirement suddenly arises? This scenario is a current significant pain point for IaaS cloud.  In such situations, the organization is typically resorting to one of two agonizingly inefficient processes. The first option involves shipping the provider hard drives for their IT staff to copy the data in bulk for download and having that data shipped back. Rackspace’s guidelines provide that a transfer of 2 terabytes of bulk files would cost over $10,000 in fees and require about four to six weeks. And then all the company gets is a full 2 terabyte duplicate of its data that still must be searched, processed and reviewed.

The other alternative is to slowly download the data through a secure file transfer protocol connection. However, even with a robust T2 line, it would take three to six weeks to transfer the two TBs, depending on how much dedicated bandwidth IT would be willing to dedicate to the exercise.

So what is needed is robust eDiscovery software that can truly support the IaaS cloud where the data resides without first requiring mass data export. We will discuss what that entails and the requirements of truly cloud capable eDiscovery software in our next post, so please stay tuned!

Leave a comment

Filed under Cloud Data, IaaS