FightForProgrammer-CSDN博客

原创 spark java.lang.OutOfMemoryError: Java heap space

spark java.lang.OutOfMemoryError: Java heap space经常遇到此问题，这个说得很详细http://stackoverflow.com/questions/21138751/spark-java-lang-outofmemoryerror-java-heap-space

2015-11-14 17:57:11 1832

原创 sql替换NOT IN语句

进来项目中用到很多sql的语句，其中有个需求：有两个表table1、table2，需要找出table1中存在，而table2中不存在的数据。要求不能使用NOT IN语句。举例：两表的数据如下：现在要从table1中选出记录，记录的f3字段不在table2中出现。一、简单的使用NOT INselect f1,f2,f3 from tabl

2015-11-12 21:52:54 3803

原创 spark问题归纳

spark问题归纳一、说明以下遇到的spark问题都是本人在实际过程中碰到的，可能并不适合所有的情况，可做参考，有异议欢迎讨论。二、问题列表2.1 Application history not found (app-20150821200352-0022) No event logs found for application in hdfs:

2015-08-24 11:05:59 985

原创欢迎使用CSDN-markdown编辑器

欢迎使用Markdown编辑器写博客本Markdown编辑器使用StackEdit修改而来，用它写博客，将会带来全新的体验哦：Markdown和扩展Markdown简洁的语法代码块高亮图片链接和图片上传LaTex数学公式UML序列图和流程图离线写博客导入导出Markdown文件丰富的快捷键快捷键加粗 Ctrl + B 斜体 Ctrl + I 引用 Ctrl

2015-08-01 17:25:28 526

原创 Spark体系及RDD

Spark体系架构Spark体系架构包括如下三个主要组件：数据存储API管理框架接下来让我们详细了解一下这些组件。数据存储：Spark用HDFS文件系统存储数据。它可用于存储任何兼容于Hadoop的数据源，包括HDFS，HBase，Cassandra等。API：利用API，应用开发者可以用标准的API接口创建基于Spark的应用。

2015-07-31 20:42:18 881

原创 Spark API

Spark RDD API使用说明（一）1、aggregate1.1 函数声明defaggregate[U: ClassTag](zeroValue: U)(seqOp: (U, T) => U, combOp: (U, U)=> U): U1.2函数说明aggregate函数通过两个函数来操作RDD。第一个reduce函数(seqOp)对每个partition聚合，然后将初

2015-07-31 20:16:38 717

原创 Spark初探

Spark初探一、Spark简介及特性1.1什么是SparkApache Spark是一个围绕速度、易用性和复杂分析构建的大数据处理框架。最初在2009年由加州大学伯克利分校的AMPLab开发，并于2010年成为Apache的开源项目之一。Spark为我们提供了一个全面、统一的框架用于管理各种有着不同性质（文本数据、图表数据等）的数据集和数据源（批量数据或实时的流数据）的大

2015-07-31 19:57:18 668

原创 Spark streaming 初探

Spark streaming 初探一、原理和运行场景Spark Streaming运行原理图如下：Spark Streaming把Kafka、HDFS、Socket等系统作为流处理的数据来源，把输入的数据流用时间切片的方式把数据分为一个个小的Batch，然后把这些小的Batch交给Spark引擎处理。官方wordcount示例代码：package o

2015-07-28 16:07:02 1007

原创 Word Ladder

Given two words (beginWord and endWord), and a dictionary, find the length of shortest transformation sequence from beginWord to endWord, such that:Only one letter can be changed at a timeEach int

2015-06-21 17:21:59 645

原创 LeetCode Word Break

LeetCode Word BreakGiven a string s and a dictionary of words dict, determine if s can be segmented into a space-separated sequence of one or more dictionary words.For example, givens = "l

2015-06-10 09:57:42 706

原创 LeetCode Maximal Square

LeetCode Maximal SquareGiven a 2D binary matrix filled with 0's and 1's, find the largest square containing all 1's and return its area.For example, given the following matrix:1 0 1 0 01

2015-06-05 15:06:40 563

原创 LeetCode Contains Duplicate II Contains Duplicate III

Contains Duplicate II Contains Duplicate III一、Contains Duplicate II Given an array of integers and an integer k, find out whether there there are two distinct indices i and j in the

2015-06-02 15:27:53 696

原创 LeetCode Course Schedule

LeetCode Course ScheduleThere are a total of n courses you have to take, labeled from 0 to n - 1.Some courses may have prerequisites, for example to take course 0 you have to first take co

2015-05-31 10:41:54 992

原创 LeetCode Word Search II

LeetCode Word Search IIGiven a 2D board and a list of words from the dictionary, find all words in the board.Each word must be constructed from letters of sequentially adjacent cell, where "

2015-05-28 14:37:06 1133

原创 LeetCode Add and Search Word - Data structure design

Add and Search Word - Data structure design Design a data structure that supports the following two operations:void addWord(word)bool search(word)search(word) can search a literal word o

2015-05-21 14:59:53 672

原创 LeetCode Implement Trie (Prefix Tree)

LeetCode Implement Trie (Prefix Tree)Implement a trie with insert, search, and startsWith methods.Note:You may assume that all inputs are consist of lowercase letters a-z.题目要求是实现字典

2015-05-19 10:30:15 621

原创 LeetCode Minimum Size Subarray Sum

LeetCode Minimum Size Subarray SumGiven an array of n positive integers and a positive integer s, find the minimal length of a subarray of which the sum ≥ s. If there isn't one, return 0 inste

2015-05-16 22:03:16 836

原创 LeetCode LRU Cache

LeetCode LRU CacheDesign and implement a data structure for Least Recently Used (LRU) cache. It should support the following operations: get and set.get(key) - Get the value (will always be

2015-05-15 15:52:09 668

原创 vs2010 error link2019无法解析的外部符号

vs2010 error link2019无法解析的外部符号一、动机自己写的Dll库，然后对其进行引用，出现了题中所示的错误。使用了以下步骤的解决办法。1、引入dll的头文件：属性-》c/c++-》常规-》附加包含目录，在框中加入dll的头文件2、引入lib库目录：属性-》链接器-》常规-》附加库目录，在框中加入生成dll对应的lib库目录（注意这里加入的是文件目录）3、引

2015-05-11 21:36:07 9663

原创 LeetCode House Robber

LeetCode House RobberYou are a professional robber planning to rob houses along a street. Each house has a certain amount of money stashed, the only constraint stopping you from robbing each of

2015-05-10 17:08:03 742

原创 Windows下C/C++连接MySQL

Windows下C/C++连接MySQL一、安装MySQL二、连接MySQL本例子使用vs2010在Win7系统下连接本机MySQL服务器。1、用vs2010创建win32控制台应用程序2、配置工程2.1、配置头文件找到MySQL安装目录下的include文件夹：打开工程配置属性窗口--> C/C++ --> 常规 --> 附加包含目录，把mysql

2015-05-09 14:10:02 1132

原创 LeetCode:Distinct Subsequences

Distinct SubsequencesGiven a string S and a string T, count the number of distinct subsequences of T in S.A subsequence of a string is a new string which is formed from the original string b

2015-04-01 18:51:34 538

原创 LeetCode : Partition List

Partition List Given a linked list and a value x, partition it such that all nodes less than x come before nodes greater than or equal to x.You should preserve the original relative order of

2015-03-31 09:29:09 557

原创 LeetCode:Construct Binary Tree from Preorder and Inorder Traversal

Construct Binary Tree from Preorder and Inorder TraversalGiven preorder and inorder traversal of a tree, construct the binary tree.Note:You may assume that duplicates do not exist in the t

2015-03-29 08:58:14 928

原创 Binary Tree Maximum Path Sum

LeetCode ：Binary Tree Maximum Path SumGiven a binary tree, find the maximum path sum.The path may start and end at any node in the tree.For example:Given the below binary tree,

2015-03-27 15:21:35 865

原创 LeetCode Min Stack

LeetCode Min StackDesign a stack that supports push, pop, top, and retrieving the minimum element in constant time.push(x) -- Push element x onto stack.pop() -- Removes the element on top of

2015-03-21 10:33:28 683

原创 LeetCode Repeated DNA Sequences

LeetCode Repeated DNA SequencesAll DNA is composed of a series of nucleotides abbreviated as A, C, G, and T, for example: "ACGAATTCCG". When studying DNA, it is sometimes useful to identify repe

2015-03-10 19:45:55 726

原创 LeetCode Rotate Array

LeetCode Rotate ArrayRotate Array Total Accepted: 5488 Total Submissions: 29811 My Submissions Question Solution Rotate an array of n elements to the right by k steps.For example, with n = 7

2015-03-03 15:16:48 1482

转载跟我一起学习VIM - The Life Changing Editor

跟我一起学习VIM - The Life Changing Editor转载出处：http://ju.outofmemory.cn/entry/79671前两天同事让我在小组内部分享一下VIM，于是我花了一点时间写了个简短的教程。虽然准备有限，但分享过程中大家大多带着一种惊叹的表情，原来编辑器可以这样强大，这算是对我多年来使用VIM的最大鼓舞吧。所以分享结束之后，将这篇

2015-01-28 12:42:20 1975

原创 Rotate Image

LeetCode Rotate ImageYou are given an n x n 2D matrix representing an image.Rotate the image by 90 degrees (clockwise).Follow up:Could you do this in-place?关键地方在于就地。而不用新的内存。

2015-01-25 20:33:04 593

原创 Gray Code

LeetCode Gray CodeThe gray code is a binary numeral system where two successive values differ in only one bit.Given a non-negative integer n representing the total number of bits in the code

2015-01-22 13:55:01 653

原创 Unique Binary Search Trees

LeetCode Unique Binary Search Trees I IIⅠ第一题主要用到了数学推导的方法。令数组arr[i]表示i个节点时BST的数目。则arr[0]=arr[1]=1，arr[2] = 2.对于已知的arr[0...n]求arr[n+1]，此n+1个节点中有个是根节点，令它的左子树个数L(n)，右子数节点个数R(n)，则左子树有arr[L(n)]中

2015-01-21 21:43:28 485

原创 Best Time to Buy and Sell Stock I II III

Best Time to Buy and Sell Stock I II IIII对于第一题，本质意义是求出数组中某两个元素（day[i],day[i]）的最大差值。但是，要求j>iint maxProfit(vector &prices) { if(prices.size() ==0) return 0;

2015-01-19 23:44:10 688

原创 LeetCode Add Two Numbers

LeetCode Add Two NumbersYou are given two linked lists representing two non-negative numbers. The digits are stored in reverse order and each of their nodes contain a single digit. Add the two n

2015-01-07 19:56:13 627

原创 LeetCode Triangle

LeetCode TriangleGiven a triangle, find the minimum path sum from top to bottom. Each step you may move to adjacent numbers on the row below.For example, given the following triangle[

2015-01-04 16:10:53 582

原创 LeetCode Recover Binary Search Tree

LeetCode Recover Binary Search TreeTwo elements of a binary search tree (BST) are swapped by mistake.Recover the tree without changing its structure.Note:A solution using O(n) space is

2015-01-01 23:58:05 573

转载 Linux 中将VIM改装成强大C的IDE

By:吴垠Date:2007-09-07Version:0.5Email:lazy.fox.wu#gmail.comHomepage:http://blog.csdn.net/wooinCopyright:该文章版权由吴垠和他可爱的老婆小包子所有。可在非商业目的下任意传播和复制。对于商业目的下对本文的任何行为需经作

2014-12-31 11:36:29 1056

原创 LeetCode Longest Palindromic Substring

LeetCode Longest Palindromic SubstringGiven a string S, find the longest palindromic substring in S. You may assume that the maximum length of S is 1000, and there exists one unique longest pa

2014-12-30 00:17:29 657

原创 LeetCode Longest Common Prefix

LeetCode Longest Common PrefixWrite a function to find the longest common prefix string amongst an array of strings.很傻逼的逐个比较法。。string longestCommonPrefix(vector &strs) { string r

2014-12-23 17:34:34 776

原创 LeetCode Valid Parentheses

LeetCode Valid ParenthesesGiven a string containing just the characters '(', ')', '{', '}', '[' and ']', determine if the input string is valid.The brackets must close in the correct order

2014-12-21 17:45:58 465

SQLite网络化工具包

空空如也