- 博客(11)
- 收藏
- 关注
原创 BigDL 运行 LeNet5 on MNIST 发现的 BUG
1. 部署Hadoop 2.8.3, Spark 2.1.02. 编译和部署BigDL 编译command: ./make-dist.sh -P spark_2.x3. 运行脚本LeNet5-on-MNIST-example.sh:#!/bin/bash$SPARK_HOME/bin/spark-submit \ --master yarn \ --deploy-mode cli...
2018-03-08 02:32:11 731
原创 LeetCode 119. Pascal's Triangle II
题目链接:https://leetcode.com/problems/pascals-triangle-ii/Given an index k, return the kth row of the Pascal's triangle.For example, given k = 3,Return [1,3,3,1].基本思路:1. 关于什么是Pa
2017-01-07 02:36:33 247
原创 LeetCode 118. Pascal's Triangle
题目链接:https://leetcode.com/problems/pascals-triangle/Given numRows, generate the first numRows of Pascal's triangle.For example, given numRows = 5,Return[ [1], [1,1], [1,2,1],
2017-01-07 02:13:09 315
原创 LeetCode 150. Evaluate Reverse Polish Notation
题目链接:https://leetcode.com/problems/evaluate-reverse-polish-notation/Evaluate the value of an arithmetic expression in Reverse Polish Notation.Valid operators are +, -, *, /. Each operand m
2017-01-04 04:29:26 273
原创 Spark1.6.3 Driver端 task运行完成源码分析
0.背景0.1当完成task调度:Executor就会收到serialized的 task,先deserialize 出正常的task,然后运行task得到结果,并通过状态更新的RPC告知SchedulerBackend0.2这里主要分析GoarseGrainedSchedulerBackend.scala的情况1.GoarseGrainedSchedule
2017-01-03 11:04:30 456
原创 Spark1.6.3 on Yarn 调度总览
先放图再分析, 估计后续的2.0版本也差不多,应该不会有大变动0. Scheduling OverView1. Stage选择DAGScheduler.scala接收到一个Job提交后,会根据computing chain和 RDDs之间的dependencies自后向前建立DAG图并划分若干stages。然后从前向后依次提交stage,因为stages之间具有依赖关系,因
2017-01-02 10:56:03 581
原创 Spark1.6.3 Driver端 task调度源码分析
0.背景当一个Action操作触发了Spark Job提交后:1. Spark首先会在DAGScheduler.scala 中对其stages进行递归分析2. 然后针对某个可执行的stage,根据其partitions的大小生产tasks,并封装成TaskSet3. 最后调用taskSchedulerImpl.scala的submitTasks(taskSet) 方法发送backe
2017-01-02 08:53:26 988
原创 LeetCode 284. Peeking Iterator
题目链接: https://leetcode.com/problems/peeking-iterator/Given an Iterator class interface with methods: next() and hasNext(), design and implement a PeekingIterator that support the peek()ope
2016-07-19 14:04:04 235
原创 LeetCode 374. Guess Number Higher or Lower
题目链接: https://leetcode.com/problems/guess-number-higher-or-lower/We are playing the Guess Game. The game is as follows:I pick a number from 1 to n. You have to guess which number I picke
2016-07-18 00:51:12 330
原创 LeetCode 62. Unique Paths
题目链接:https://leetcode.com/problems/unique-paths/A robot is located at the top-left corner of a m x n grid (marked 'Start' in the diagram below).The robot can only move either down or right at
2016-07-17 04:12:59 194
原创 Linux下, Hadoop2.6.0源代码导入Eclipse
环境:Linux: Ubuntu14.04 Java: JDK1.7 Eclipse: eclipse-standard-kepler-SR2-linux-gtk-x86_64.tar.gz Hadoop:version-2.6.0 Maven: 3.0.51.安装JDK,Eclipse,Maven请自行百度,下
2016-01-14 15:30:00 1148
空空如也
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人