Aggregating geographically sparse feeds onto a single API
Gwen Bart handles the IT infrastructure and implementation at a very renowned Social Media Intelligence House. His agency had been growing at a rapid pace recently and planned to expand to different geographic locations soon. A major chunk of their work revolved around acquiring data feeds and social media data from nearby geographic locations and analysing them to give a better understanding of the conversations revolving around a set of keywords in specific areas.
They wanted to acquire such relevant data from more than 5000 sources in a structured collated format which could be simply imported onto their data platform. Also, they wanted this data to be searchable for more than 1000 set of specific keywords and phrases or for specific queries. Earlier Gwen had tried building such a customised crawler in house and extracting relevant data feeds, but the geographic location associated with feeds was proving incorrect in numerous cases and the pressure to fetch accurate data was mounting. Gwen was facing a serious pressure for lack of resources and time. Thus, he decided to outsource this requirement so he could focus on maintaining his end of the platform that was a crucial infrastructure their data analytics service stood upon.
How did PromptCloud Help?
- PromptCloud set up a mass scale crawl which enabled crawling numerous sources in parallel at regular periodic intervals in a day.
- Feeds from various social media were aggregated intelligently by developing a Geo-Intelligence API which helped capture feeds only from desired geographic locations.
- List of locations, sources, keywords and queries was dynamically modified based upon the client requirements and feedback.
- Over 2,00,000 feeds were collected from various continents within 2 months of time.
- Every week fresh data is collated location-wise and delivered.
Benefits to the client
- Parallel collection of data from numerous sources without any infrastructural concerns
- Uniform Data schema irrespective of number of sources and heterogeneity of content
- Periodic delivery of fresh data, reducing further data processing efforts
- Geo-Intelligence API assuring data to be belonging to the described geography
- Scalable solution with increasing number of sources, locations and keywords
Thus, we helped Gwen create a reliable source of accurate data feeds which were easily integrated with his analytics platform in no time.
Have any such requirements for geography specific data feeds Reach out to us at firstname.lastname@example.org or just Get Started