WepresentMesos,aplatformforsharingcommodityclustersbetweenmultiplediverseclustercomputingframeworks,suchasHadoopandMPI.Sharingimprovesclusterutilizationandavoidsper-frameworkdatareplication.Mesossharesresourcesinafine-grainedmanner,allowingframeworkstoachievedatalocalitybytakingturnsreadingdatastoredoneachmachine.Tosupportthesophisticatedschedulersoftoday’sframeworks,Mesosintroducesadistributedtwo-levelschedulingmechanismcalledresourceoffers.Mesosdecideshowmanyresourcestooffereachframework,whileframeworksdecidewhichresourcestoacceptandwhichcomputationstorunonthem.OurresultsshowthatMesoscanachievenear-optimaldatalocalitywhensharingtheclusteramongdiverseframeworks,canscaleto50,000(emulated)nodes,andisresilienttofailures.
2025/2/19 11:57:02
829KB
mesos
1