Management’s Two Key Concerns
Business managers need to address the following two concerns, Demonstration of Value and Operationalization.
Demonstration of Value
This is to channel the energy and effort of test-driving big data technologies in the context of one or more real business problems to determine whether those technologies add value.
To devise a communication strategy for sharing the message to the right people within the organization to identify a champion, secure business sponsorship and establish bidirectional engagement to keep business sponsors educated and aware of progress.
Operationalization
To develop the tactics for the information technologists to work with the data management professionals and the business stakeholders to migrate the big data projects into the production environment in a controlled and managed way, without disrupting day-to-day operations.
Historical Perspective to Reporting and Analytics
The data warehouse model has undergone a series of refinements in reaction to a continuous need for increased performance and shortened cycles for delivering actionable knowledge.
It is largely focused on providing reports from transactional systems without interfering with the operational performance of those systems.
Transaction processing data models and systems are engineer to accommodate updates to a few records at a time.
Transaction processing application systems are optimized to enable short response time to facilitate user productivity.
Data Warehouse and Data Marts
The data warehouse model is a centralized repository of data that has been extracted from operational systems and transformed into a structure that facilitates the analytical processing of the data.
Data marts are smaller, subject-oriented data warehouses that are focused on a specific business function or subject area.
Historical Perspective to Reporting and Analytics (Cont.)
Conventional strategy is information fragmentation and reorganization.
- Datasets are extracted from the source systems and loaded into online analytical processing (OLAP) systems or other analytical appliances and tuned to meet the anticipated performance needs for analytical queries.
- At the same time, datasets are replicated according to geography to fitness data latency and avoid network bottlenecks.
- However, such topology has unnecessarily complicated the information architecture and artificially impedes the ability to provide real-time reporting and analytics.
Considerations of Adopting Big Data Technologies
- Steep learning curve
- Developers have to have some experience in data distribution and high performance / parallel code development.
- Data life cycle changes
- BDA is different from traditional transaction processing which deliver results based on static structured datasets.
- Existing infrastructure
- Data warehouse vs federation & virtualization which allow data to remain in its original source.
- Existing investments
- Significant investments have been done in developing BI / DW (Business Intelligence / Data Warehouse) environments.
- Data Intent
- Most data instances in traditional systems are created for specific purposes while BD applications seek to repurpose data for analysis.
- Size and Duration
- The transitory characteristics associated with rapid turnaround of numerous data streams conflicts with the desire to retain very large datasets in anticipation of the potential for new analyses to be performed in the future.
Involving the Right Decision Makers
It is incumbent upon the key stakeholders in the organizations to make sure that the business process owners, the information consumers, the technical infrastructure innovators, the application developers, and the enterprise architects all work together in an environment that can continue to satisfy existing reporting needs yet is flexible enough for exploratory work.
Roles of Organizational Alignment
- Business Evangelist
- Understands the types of performance barriers imposed by the existing technical infrastructure.
- Understands the ongoing reviews of emerging technology may create efficiencies that do not currently exist within the organization.
- Socialize the value of exploring the use of new techniques among the business process owners and to solicit their input to understand their current and future needs to guide the selection of technologies to review and possibly pilot.
- Technical Evangelist
- Understands the emerging technology and the science behind new methods, where the technology can potentially improve the application environment, either by improving the performance of existing processes or by enabling new capabilities.
- Business Analyst
- Engages the business process owners and solicits their needs and expectations for identifying some of the key quantifiable measures for evaluating the business benefits of the new technology as well as frames the technical requirements for any pilot project.
- Big Data Application Architect
- Making sure that any pilot development is designed with reasonable experience in performance computing.
- Application Developer
- Identify the technical resources with the right set of skills for programming and testing parallel and distributed applications.
- Program Manager
- Plans and oversees any pilot development to make sure that it remains aligned with organizational expectations, remains within an allocated budged, and is properly documented to ensure that the best practices can be captured and migrated into production.
Conclusion
The alignment of business and technology is critical to the success of any big data analytics project. The business process owners must be engaged in the process to ensure that the technology is being applied to the right problems and that the results are being communicated in a way that is meaningful to the business.
The technical infrastructure innovators must be engaged to ensure that the technology is being applied in a way that is scalable, reliable, and secure. The application developers must be engaged to ensure that the technology is being applied in a way that is efficient, maintainable, and extensible.
The enterprise architects must be engaged to ensure that the technology is being applied in a way that is aligned with the overall architecture of the organization. The program managers must be engaged to ensure that the technology is being applied in a way that is aligned with the overall goals of the organization and that the project is being managed in a way that is efficient and effective.