Did you know that there are 12 factors to be considered while acquiring data from the web? If no, fret not! Download our free guide on web data acquisition to get started!
Modern Big Data architecture is fast changing the world with its rapid pace and seamless usage it offers to the users. As we read in Part-1 and Part-2 of this 3 part series on Big Data. It has come a long way from its inception way back from the 1600s to recent modern-day Big Data. Data Mining and Big Data has come a long way since then and has changed the way people look at technology.
The effects of the combined events described in part 1 of this blog series, led to the creation of NSA (National Security Agency) of the USA by then-President Truman. The key activities of the staffers of NSA are to decrypt the messaged intercepted during the Cold War.
The National Security Agency of the United States of America
In 1969, the Advanced Research Projects Agency, a subdivision of Department of Defence, USA, funded a project called ARPANET which began when a message was sent from a host computer in UCLA to a host computer in Stanford. This is the birth of the Internet as we know it today. But later in 1989, the architecture started to age. Many organizations shifted from ARPANET to other Networks like NSFNET to increase efficiency and speed.
The first data center built by the US Government to store millions of fingerprints sets and tax returns. These records stored in magnetic tape and stored in a central location. But the project was closed due to the expressed concerns by various segments of people. This was the first attempt at a large-scale data storage.
In 1977, microcomputers called Personal Computer introduced to the market. This is yet another milestone in the evolution of the internet and eventually the Big Data. The personal computers were much simpler and more accessible than a mainframe computer which required a large processor shared by multiple individuals and an operating staff or a system for shared access. Many of the early personal computers sold as electronic kits for hobbyists and enthusiasts.
A British scientist called Tim Berners-Lee working in CERN (European Council for Nuclear Research) invented the World Wide Web. His goal was to share information on the internet using the hypertext system which allowed the transfer of not just text data but also audio and video data as well. In 1993, The CERN announced the WWW. The key factor was that it was free for everyone to develop and use. The increase in the power and flexibility of personal computers and free access to WWW contributed to exponential growth on the internet in the following years.
In 2005, Hadoop created to handle modern Big Data architecture. This was based on an open-source software platform called Nutch. Because of its ability to process structured and unstructured data from almost all the digital resources and its flexibility HADOOP was able to process Big Data.
The internet and the WWW are the founding pillars for the modern Big Data. The immense volume of data is being derived from multiple ways such as the billions of websites, sensors, IoT, etc. the process of getting data from websites to use for analysis and other purposes is called Web Scraping. PromptCloud Technologies is a pioneer and an industry leader when it comes to web scraping services.
Your email address will not be published. Required fields are marked *
Save my name, email, and website in this browser for the next time I comment.