2022 |
Vogel, Adriano; Griebler, Dalvan; Danelutto, Marco; Fernandes, Luiz Gustavo Self-adaptation on Parallel Stream Processing: A Systematic Review Journal Article doi Concurrency and Computation: Practice and Experience, 34 (6), pp. e6759, 2022. @article{VOGEL:Survey:CCPE:2022, title = {Self-adaptation on Parallel Stream Processing: A Systematic Review}, author = {Adriano Vogel and Dalvan Griebler and Marco Danelutto and Luiz Gustavo Fernandes}, url = {https://doi.org/10.1002/cpe.6759}, doi = {10.1002/cpe.6759}, year = {2022}, date = {2022-03-01}, journal = {Concurrency and Computation: Practice and Experience}, volume = {34}, number = {6}, pages = {e6759}, publisher = {Wiley}, abstract = {A recurrent challenge in real-world applications is autonomous management of the executions at run-time. In this vein, stream processing is a class of applications that compute data flowing in the form of streams (e.g., video feeds, images, and data analytics), where parallel computing can help accelerate the executions. On the one hand, stream processing applications are becoming more complex, dynamic, and long-running. On the other hand, it is unfeasible for humans to monitor and manually change the executions continuously. Hence, self-adaptation can reduce costs and human efforts by providing a higher-level abstraction with an autonomic/seamless management of executions. In this work, we aim at providing a literature review regarding self-adaptation applied to the parallel stream processing domain. We present a comprehensive revision using a systematic literature review method. Moreover, we propose a taxonomy to categorize and classify the existing self-adaptive approaches. Finally, applying the taxonomy made it possible to characterize the state-of-the-art, identify trends, and discuss open research challenges and future opportunities.}, keywords = {}, pubstate = {published}, tppubtype = {article} } A recurrent challenge in real-world applications is autonomous management of the executions at run-time. In this vein, stream processing is a class of applications that compute data flowing in the form of streams (e.g., video feeds, images, and data analytics), where parallel computing can help accelerate the executions. On the one hand, stream processing applications are becoming more complex, dynamic, and long-running. On the other hand, it is unfeasible for humans to monitor and manually change the executions continuously. Hence, self-adaptation can reduce costs and human efforts by providing a higher-level abstraction with an autonomic/seamless management of executions. In this work, we aim at providing a literature review regarding self-adaptation applied to the parallel stream processing domain. We present a comprehensive revision using a systematic literature review method. Moreover, we propose a taxonomy to categorize and classify the existing self-adaptive approaches. Finally, applying the taxonomy made it possible to characterize the state-of-the-art, identify trends, and discuss open research challenges and future opportunities. |
Müller, Caetano; Löff, Junior; Griebler, Dalvan; Eizirik, Eduardo Avaliação da aplicação de paralelismo em classificadores taxonômicos usando Qiime2 Inproceedings doi Anais da XXII Escola Regional de Alto Desempenho da Região Sul, pp. 25–28, Sociedade Brasileira de Computação (SBC), Porto Alegre, RS, Brasil, 2022. @inproceedings{gmap:MULLER:ERAD-RS:22, title = {Avaliação da aplicação de paralelismo em classificadores taxonômicos usando Qiime2}, author = {Caetano Müller and Junior Löff and Dalvan Griebler and Eduardo Eizirik}, url = {https://sol.sbc.org.br/index.php/eradrs/article/view/19152}, doi = {10.5753/eradrs.2022.19152}, year = {2022}, date = {2022-01-01}, booktitle = {Anais da XXII Escola Regional de Alto Desempenho da Região Sul}, pages = {25--28}, publisher = {Sociedade Brasileira de Computação (SBC)}, address = {Porto Alegre, RS, Brasil}, abstract = {A classificação de sequências de DNA usando algoritmos de aprendizado de máquina ainda tem espaço para evoluir, tanto na qualidade do resultado quanto na eficiência computacional dos algoritmos. Nesse trabalho, realizou-se uma avaliação de desempenho em dois algoritmos de aprendizado de máquina da ferramenta Qiime2 para classificação de sequências de DNA. Os resultados mostram que o desempenho melhorou em até 9,65 vezes utilizando 9 threads.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } A classificação de sequências de DNA usando algoritmos de aprendizado de máquina ainda tem espaço para evoluir, tanto na qualidade do resultado quanto na eficiência computacional dos algoritmos. Nesse trabalho, realizou-se uma avaliação de desempenho em dois algoritmos de aprendizado de máquina da ferramenta Qiime2 para classificação de sequências de DNA. Os resultados mostram que o desempenho melhorou em até 9,65 vezes utilizando 9 threads. |
Garcia, Adriano Marques; Griebler, Dalvan; Schepke, Claudio; Fernandes, Luiz Gustavo SPBench: a framework for creating benchmarks of stream processing applications Journal Article doi Computing, In press (In press), pp. 1-23, 2022. @article{GARCIA:Computing:22, title = {SPBench: a framework for creating benchmarks of stream processing applications}, author = {Adriano Marques Garcia and Dalvan Griebler and Claudio Schepke and Luiz Gustavo Fernandes}, url = {https://doi.org/10.1007/s00607-021-01025-6}, doi = {10.1007/s00607-021-01025-6}, year = {2022}, date = {2022-01-01}, journal = {Computing}, volume = {In press}, number = {In press}, pages = {1-23}, publisher = {Springer}, abstract = {In a fast-changing data-driven world, real-time data processing systems are becoming ubiquitous in everyday applications. The increasing data we produce, such as audio, video, image, and, text are demanding quickly and efficiently computation. Stream Parallelism allows accelerating this computation for real-time processing. But it is still a challenging task and most reserved for experts. In this paper, we present SPBench, a framework for benchmarking stream processing applications. It aims to support users with a set of real-world stream processing applications, which are made accessible through an Application Programming Interface (API) and executable via Command Line Interface (CLI) to create custom benchmarks. We tested SPBench by implementing parallel benchmarks with Intel Threading Building Blocks (TBB), FastFlow, and SPar. This evaluation provided useful insights and revealed the feasibility of the proposed framework in terms of usage, customization, and performance analysis. SPBench demonstrated to be a high-level, reusable, extensible, and easy of use abstraction to build parallel stream processing benchmarks on multi-core architectures.}, keywords = {}, pubstate = {published}, tppubtype = {article} } In a fast-changing data-driven world, real-time data processing systems are becoming ubiquitous in everyday applications. The increasing data we produce, such as audio, video, image, and, text are demanding quickly and efficiently computation. Stream Parallelism allows accelerating this computation for real-time processing. But it is still a challenging task and most reserved for experts. In this paper, we present SPBench, a framework for benchmarking stream processing applications. It aims to support users with a set of real-world stream processing applications, which are made accessible through an Application Programming Interface (API) and executable via Command Line Interface (CLI) to create custom benchmarks. We tested SPBench by implementing parallel benchmarks with Intel Threading Building Blocks (TBB), FastFlow, and SPar. This evaluation provided useful insights and revealed the feasibility of the proposed framework in terms of usage, customization, and performance analysis. SPBench demonstrated to be a high-level, reusable, extensible, and easy of use abstraction to build parallel stream processing benchmarks on multi-core architectures. |
2021 |
Löff, Júnior; Hoffmann, Renato Barreto; Griebler, Dalvan; Fernandes, Luiz G High-Level Stream and Data Parallelism in C++ for Multi-Cores Inproceedings doi XXV Brazilian Symposium on Programming Languages (SBLP), pp. 41-48, ACM, Joinville, Brazil, 2021. @inproceedings{LOFF:SBLP:21, title = {High-Level Stream and Data Parallelism in C++ for Multi-Cores}, author = {Júnior Löff and Renato Barreto Hoffmann and Dalvan Griebler and Luiz G Fernandes}, url = {https://dl.acm.org/doi/10.1145/3475061.3475078}, doi = {10.1145/3475061.3475078}, year = {2021}, date = {2021-09-01}, booktitle = {XXV Brazilian Symposium on Programming Languages (SBLP)}, pages = {41-48}, publisher = {ACM}, address = {Joinville, Brazil}, series = {SBLP'21}, abstract = {Stream processing applications have seen an increasing demand with the increased availability of sensors, IoT devices, and user data. Modern systems can generate millions of data items per day that require to be processed timely. To deal with this demand, application programmers must consider parallelism to exploit the maximum performance of the underlying hardware resources. However, parallel programming is often difficult and error-prone, because programmers must deal with low-level system and architecture details. In this work, we introduce a new strategy for automatic data-parallel code generation in C++ targeting multi-core architectures. This strategy was integrated with an annotation-based parallel programming abstraction named SPar. We have increased SPar’s expressiveness for supporting stream and data parallelism, and their arbitrary composition. Therefore, we added two new attributes to its language and improved the compiler parallel code generation. We conducted a set of experiments on different stream and data-parallel applications to assess the efficiency of our solution. The results showed that the new SPar version obtained similar performance with respect to handwritten parallelizations. Moreover, the new SPar version is able to achieve up to 74.9x better performance with respect to the original ones due to this work.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Stream processing applications have seen an increasing demand with the increased availability of sensors, IoT devices, and user data. Modern systems can generate millions of data items per day that require to be processed timely. To deal with this demand, application programmers must consider parallelism to exploit the maximum performance of the underlying hardware resources. However, parallel programming is often difficult and error-prone, because programmers must deal with low-level system and architecture details. In this work, we introduce a new strategy for automatic data-parallel code generation in C++ targeting multi-core architectures. This strategy was integrated with an annotation-based parallel programming abstraction named SPar. We have increased SPar’s expressiveness for supporting stream and data parallelism, and their arbitrary composition. Therefore, we added two new attributes to its language and improved the compiler parallel code generation. We conducted a set of experiments on different stream and data-parallel applications to assess the efficiency of our solution. The results showed that the new SPar version obtained similar performance with respect to handwritten parallelizations. Moreover, the new SPar version is able to achieve up to 74.9x better performance with respect to the original ones due to this work. |
Andrade, Gabriella; Griebler, Dalvan; Santos, Rodrigo; Danelutto, Marco; Fernandes, Luiz Gustavo Assessing Coding Metrics for Parallel Programming of Stream Processing Programs on Multi-cores Inproceedings doi 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 291-295, IEEE, Pavia, Italy, 2021, ISBN: 978-1-6654-2705-0. @inproceedings{ANDRADE:SEAA:21, title = {Assessing Coding Metrics for Parallel Programming of Stream Processing Programs on Multi-cores}, author = {Gabriella Andrade and Dalvan Griebler and Rodrigo Santos and Marco Danelutto and Luiz Gustavo Fernandes}, url = {https://doi.org/10.1109/SEAA53835.2021.00044}, doi = {10.1109/SEAA53835.2021.00044}, isbn = {978-1-6654-2705-0}, year = {2021}, date = {2021-09-01}, booktitle = {2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)}, pages = {291-295}, publisher = {IEEE}, address = {Pavia, Italy}, series = {SEAA'21}, abstract = {From the popularization of multi-core architectures, several parallel APIs have emerged, helping to abstract the programming complexity and increasing productivity in application development. Unfortunately, only a few research efforts in this direction managed to show the usability pay-back of the programming abstraction created, because it is not easy and poses many challenges for conducting empirical software engineering. We believe that coding metrics commonly used in software engineering code measurements can give useful indicators on the programming effort of parallel applications and APIs. These metrics were designed for general purposes without considering the evaluation of applications from a specific domain. In this study, we aim to evaluate the feasibility of seven coding metrics to be used in the parallel programming domain. To do so, five stream processing applications implemented with different parallel APIs for multi-cores were considered. Our experiments have shown COCOMO II is a suitable model for evaluating the productivity of different parallel APIs targeting multi-cores on stream processing applications while other metrics are restricted to the code size.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } From the popularization of multi-core architectures, several parallel APIs have emerged, helping to abstract the programming complexity and increasing productivity in application development. Unfortunately, only a few research efforts in this direction managed to show the usability pay-back of the programming abstraction created, because it is not easy and poses many challenges for conducting empirical software engineering. We believe that coding metrics commonly used in software engineering code measurements can give useful indicators on the programming effort of parallel applications and APIs. These metrics were designed for general purposes without considering the evaluation of applications from a specific domain. In this study, we aim to evaluate the feasibility of seven coding metrics to be used in the parallel programming domain. To do so, five stream processing applications implemented with different parallel APIs for multi-cores were considered. Our experiments have shown COCOMO II is a suitable model for evaluating the productivity of different parallel APIs targeting multi-cores on stream processing applications while other metrics are restricted to the code size. |
Parallel Applications Modelling Group
GMAP is a research group at the Pontifical Catholic University of Rio Grande do Sul (PUCRS). Historically, the group has conducted several types of research on modeling and adapting robust, real-world applications from different domains (physics, mathematics, geology, image processing, biology, aerospace, and many others) to run efficiently on High-Performance Computing (HPC) architectures, such as Clusters.
In the last decade, new abstractions of parallelism are being created through domain-specific languages (DSLs), libraries, and frameworks for the next generation of computer algorithms and architectures, such as embedded hardware and servers with accelerators like Graphics Processing Units (GPUs) or Field-Programmable Gate Array (FPGAs). This has been applied to stream processing and data science-oriented applications. Concomitantly, since 2018, research is being conducted using artificial intelligence to optimize applications in the areas of Medicine, Ecology, Industry, Agriculture, Education, Smart Cities, and others.
Research Lines
Applied Data Science
Parallelism Abstractions
The research line HSPA (High-level and Structured Parallelism Abstractions) aims to create programming interfaces for the user/programmer who is not able in dealing with the parallel programming paradigm. The idea is to offer a higher level of abstraction, where the performance of applications is not compromised. The interfaces developed in this research line go toward specific domains that can later extend to other areas. The scope of the study is broad as regards the use of technologies for the development of the interface and parallelism.
Parallel Application Modeling
Team

Prof. Dr. Luiz Gustavo Leão Fernandes
General Coordinator

Prof. Dr. Dalvan Griebler
Research Coordinator
Last Papers
2022 |
Self-adaptation on Parallel Stream Processing: A Systematic Review Journal Article doi Concurrency and Computation: Practice and Experience, 34 (6), pp. e6759, 2022. |
Avaliação da aplicação de paralelismo em classificadores taxonômicos usando Qiime2 Inproceedings doi Anais da XXII Escola Regional de Alto Desempenho da Região Sul, pp. 25–28, Sociedade Brasileira de Computação (SBC), Porto Alegre, RS, Brasil, 2022. |
SPBench: a framework for creating benchmarks of stream processing applications Journal Article doi Computing, In press (In press), pp. 1-23, 2022. |
2021 |
High-Level Stream and Data Parallelism in C++ for Multi-Cores Inproceedings doi XXV Brazilian Symposium on Programming Languages (SBLP), pp. 41-48, ACM, Joinville, Brazil, 2021. |
Assessing Coding Metrics for Parallel Programming of Stream Processing Programs on Multi-cores Inproceedings doi 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 291-295, IEEE, Pavia, Italy, 2021, ISBN: 978-1-6654-2705-0. |
Projects
Software
Last News
Contact us!
Or, feel free to use the form below to contact us.