论文标题

考虑通信和IO成本的新模型,用于大规模平行计算

A New Model for Massively Parallel Computation Considering both Communication and IO Cost

论文作者

Ma, Hengzhao, Gao, Xiangyu, Li, Jianzhong, Gao, Tianpeng

论文摘要

在平行计算的研究领域中,对通信成本进行了广泛的研究,而IO成本已被忽略。对于大数据计算,必须将数据拟合到主内存中不再保留,并且必须使用外部内存。因此,有必要将IO成本带入并行计算模型中。在本文中,我们提出了第一个并行计算模型,该模型需要考虑IO成本以及不均匀的通信成本。根据新模型,我们提出了一些新问题,旨在最大程度地降低新模型的IO和通信成本。我们证明了这些新问题的硬度,然后设计和分析解决方案的近似算法。

In the research area of parallel computation, the communication cost has been extensively studied, while the IO cost has been neglected. For big data computation, the assumption that the data fits in main memory no longer holds, and external memory must be used. Therefore, it is necessary to bring the IO cost into the parallel computation model. In this paper, we propose the first parallel computation model which takes IO cost as well as non-uniform communication cost into consideration. Based on the new model, we raise several new problems which aim to minimize the IO and communication cost on the new model. We prove the hardness of these new problems, then design and analyze the approximate algorithms for solving them.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源