DataAnalyticswithSparkUsingPython(Addison-WesleyData&AnalyticsSeries)By作者:JeffreyAvenISBN-10书号:013484601XISBN-13书号:9780134846019Edition版本:1出版日期:2018-06-16pages页数:851SolveDataAnalyticsProblemswithSpark,PySpark,andRelatedOpenSourceToolsSparkisattheheartoftoday’sBigDatarevolution,helpingdataprofessionalssuperchargeefficiencyandperformanceinawiderangeofdataprocessingandanalyticstasks.Inthisguide,BigDataexpertJeffreyAvencoversallyouneedtoknowtoleverageSpark,togetherwithitsextensions,subprojects,andwiderecosystem.Avencombinesalanguage-agnosticintroductiontofoundationalSparkconceptswithextensiveprogrammingexamplesutilizingthepopularandintuitivePySparkdevelopmentenvironment.Thisguide’sfocusonPythonmakesitwidelyaccessibletolargeaudiencesofdataprofessionals,analysts,anddevelopers—eventhosewithlittleHadooporSparkexperience.Aven’sbroadcoveragerangesfrombasictoadvancedSparkprogramming,andSparkSQLtomachinelearning.You’lllearnhowtoefficientlymanageallformsofdatawithSpark:streaming,structured,semi-structured,andunstructured.Throughout,concisetopicoverviewsquicklygetyouuptospeed,andextensivehands-onexercisesprepareyoutosolverealproblems.Coverageincludes:UnderstandSpark’sevolvingroleintheBigDataandHadoopecosystemsCreateSparkclustersusingvariousdeploymentmodesControlandoptimizetheoperationofSparkclustersandapplicationsMasterSparkCoreRDDAPIprogrammingtechniquesExtend,accelerate,andoptimizeSparkroutineswithadvancedAPIplatformconstructs,includingsharedvariables,RDDstorage,andpartitioningEfficientlyintegrateSparkwithbothSQLandnonrelationaldatastoresPerformstreamprocessingandmessagingwithSparkStreamingandApacheKafkaImplementpredictivemodelingwithSparkRandSparkMLlibI:SparkFoundations1IntroducingBigData,Hadoop,an

Spark零基础思维导图(内含spark-core，spark-streaming,spark-sql)

spark-streaming-kafka-0-8_2.11-2.4.0.jar

HTTP_Adaptive_Streaming_QoE_Estimation_with_P.1203–Open_Databases_and_Software

xilliixpciedma驱动（基于xilnxxdmaip核4.0的WDF驱动）---#XDMAWindowsDriverThisprojectisXilinx'ssampleWindowsdriverfor'DMA/BridgeSubsystemforPCIExpressv4.0'(XDMA)IP.*Pleasenotethatthisdriverandassociatedsoftwarearesuppliedtogiveabasicgenericreferenceimplementationonly.Customersmayhavespecificuse-casesand/orrequirementsforwhichthisdriverisnotsuitable.*###Dependencies*TargetmachinerunningWindows7orWindows10*DevelopmentmachinerunningWindows7(orlater)*VisualStudio2015(orlater)installedondevelopmentmachine*WindowsDriverKit(WDK)version1703(orlater)installedondevelopmentmachine##DirectoryStructure```/|__build/-Generateddirectorycontainingbuildoutputbinaries.|__exe/-Containssampleclientapplicationsourcecode.||__simple_dma/-SamplecodeforAXI-MMconfiguredXDMAIP.||__streaming_dma/-SamplecodeforAXI-STconfiguredXDMAIP.||__user_events/-Samplecodeforaccesstousereventinterrupts.||__xdma_info/-UtilityapplicationwhichprintsouttheXDMAcoreip||configuration.||__xdma_rw/-Utilityforreading/writingto/fromxdmadevicenodessuch||ascontrol,user,bypass,h2c_0,c2h_0etc.||__xdma_test/-BasictestapplicationwhichperformsH2C/C2Htransferson|allpresentchannels.|__inc/-ContainspublicAPIheaderfileforXDMAdriver.|__libxdma/-StatickernellibraryforXDMAIP.|__sys/-Referencedriversourcecodewhichuseslibxdma|__README.md-Thisfile.|__XDMA.sln-VisualStudioSolution.```

EduCoder网站的Spark的SQL，Streaming，RDD部分。
Scala全部

flink1.9版本自带的wordcount例子，位于flink-1.9.0\examples\streaming目录下

超级详细的spark思维导图,其中包含了spark-core、spark-streaming、spark-sql...

StructuredStreaming是一个可拓展，容错的，基于SparkSQL执行引擎的流处理引擎。
使用小量的静态数据模拟流处理。
伴随流数据的到来，SparkSQL引擎会逐渐连续处理数据并且更新结果到最终的Table中。
你可以在SparkSQL上引擎上使用DataSet/DataFrameAPI处理流数据的聚集，事件窗口，和流与批次的连接操作等。
最后StructuredStreaming系统快速，稳定，端到端的恰好一次保证，支持容错的处理。

Spark-streaming在京东的项目实践，是SparkStreaming的一个很成功的使用

在日常工作中，钉钉打卡成了我生活中不可或缺的一部分。然而，有时候这个看似简单的任务却给我带来了不少烦恼。 每天早晚，我总是得牢记打开钉钉应用，点击"工作台"，再找到"考勤打卡"进行签到。有时候因为工作忙碌，会忘记打卡，导致考勤异常，影响当月的工作评价。而且，由于我使用的是苹果手机，有时候系统更新后，钉钉的某些功能会出现异常，使得打卡变得更加麻烦。 另外，我的家人使用的是安卓手机，他们也经常抱怨钉钉打卡的繁琐。尤其是对于那些不太熟悉手机操作的长辈来说，每次打卡都是一次挑战。他们总是担心自己会操作失误，导致打卡失败。 为了解决这些烦恼，我开始思考是否可以通过编写一个全自动化脚本来实现钉钉打卡。经过一段时间的摸索和学习，我终于成功编写出了一个适用于苹果和安卓系统的钉钉打卡脚本。