**星空下的小小心灵3**: The Evolution of Data Management Technology : From Traditional File to Data Warehouse

First, let us introduce you to the earliest form of data management, that is, the traditional file processing. This type of processing is a system in which data are organized, stored and processed in independent files of data records. The information is saved in several different files, each organized in a different way. Each of these files has been organised to be used by different application programs, none of which we believe, produces the information we look for in the form that we desire. It is understandable that no application programs are available to help users in getting the information we desire from these files.

Some of the drawbacks of traditional file processing include:

Data Redundancy - Where independent data files had included a massive number of duplicated data
Lack of Data Integration - Where it is tough to provide end users with information for their ad hoc requests, which as a result, require them to access data stored in several different files
Data Dependence - Where all of the processes to access files depend on each other
Lack of Data Integrity or Standardization - Where different end users and applications have to define data elements differently

A type of traditional file processing method known as the Grid-oriented Storage (GOS)

Next, we look into how traditional file processing has evolve into database management approach. According to O'Brien in Introduction to Information Systems (14th ed), database is the language and graphical tools to define entities, relationships, integrity constraints and authorization rights.

Database Developement:

Database development involves defining and organizing the content, relationships and structure of the data needed to build a database.
It consolidates data into databases that can be accessed by differently programs.
It also involves using a database management system (DBMS) which serves as a software interface between users and databases to develop prototypes of queries, forms, reports and web pages for a proposed business application.

There are 4 types of databases mainly:

Operational Databases

These databases store detailed data needed to support the operations of an entire organization.
They are also called subject-area databases (SADB), transaction databases, and production databases. For instance, customer databases, personal database, inventory database and accounting databases
It stores detailed data needed to support the business processes and operations of a company

Distributed Databases

These are databases of local work-groups and departments at regional offices, branch offices, manufacturing plants and other work sites.
It can include segments of both common operational and common user databases, as well as data generated and used only at a user’s own site.
It is the copies or parts of databases to network servers at a variety of locations

External Databases

These databases provide access to external, privately-owned data online - available for a fee to end-users and organizations from commercial services.
Access to a wealth of information from external database is available for a fee from commercial online services and with or without charge from many sources in the Internet.

Hypermedia Databases

These are a set of interconnected multimedia pages at a web-site.
They consist of a home page and other hyperlinked pages of multimedia or mixed media such as text, graphic, photographic images, video clips, audio etc.

A screenshot of a database management approach

Then, this technology evolved into something called the data warehouse. Data warehouse stores data that has been extracted from various database of an organisation. Besides storing data from previous and current years, it also acts as the central source of the data that has been cleaned, transformed, and cataloged to be able to function in data mining, analytical processing, market research and of course, for decision support too. Data warehouses are characterized by being slow to insert into but fast to retrieve from.

Some of the advantages of data warehouses:

Speed and Reliability: The physical nature of a data warehouse allows for the creation of summary and aggregate structures. When combined with indexing strategies and database tuning this results in fast and predictable analysis.
Efficiency: The data warehouse automates the repetitive data consolidation tasks associated with manual consolidation. This leaves the analysts with time to do their real job, which is to analyze.
Reliability: The data warehouse is a "single version of the truth". Typically with manual consolidation, multiple versions of these Excel spreadsheets proliferate, often with conflicting data leading to a considerable risk of business decisions being made based on incorrect information.