Towards automatic infrastructure provisioning for highly dynamic streaming applications

Publikationstyp
Konferenz
Autor(en)
P. Zehnder and D. Riemer
Jahr
2017
Seiten
4869-4871
Monat
Dec
Buchtitel
2017 IEEE International Conference on Big Data (Big Data)
Abstract
In recent years, there has been an increasing need for flexible stream processing platforms that focus on enabling domain experts to model data processing pipelines using graphical user interfaces. Such pipelines can then be automatically deployed and executed in a distributed big data infrastructure. However, a still remaining problem of such stream systems is the manual management of resources, which could be automated by adding an infrastructure manager. The manager uses available information about running pipelines, especially their semantics. This information combined with the real-time information of the underlying services executing pipelines can be used to automate the configuration of modern cluster management tools. With those tools, it is possible to add new resources into the infrastructure of existing systems and to allocate the running application accordingly. This paper presents a novel approach how information about the application and its usage can be used to adapt the infrastructure, which further reduces the manual configuration effort of such systems.
DOI
10.1109/BigData.2017.8258571
Download .bib
Download .bib
Eingetragen von
Philipp Zehnder