- 博客(114)
- 收藏
- 关注
RSS订阅原创 Pytorch使用DDP加载预训练权重时出现占用显存的多余进程
感谢知乎作者 https://www.zhihu.com/question/67209417/answer/866488638在使用DDP进行单机多卡分布式训练时，出现了在加载预训练权重时显存不够的现象，但是相同的代码单机单卡运行并不会出现问题，后来发现是在多卡训练时，额外出现了3个进程同时占用了0卡的部分显存导致的，而这3个进程正是另外3张卡load进来的数据，默认这些数据被放在了0卡上。解决的方法是把load进来的数据放在cpu（也就是内存）里。# 原来代码，load进的数据放在gpu里# pre
2021-09-16 20:56:41
1979
1
原创 解决GitHub访问慢
问题描述git clone 非常慢或者根本链接不上git clone https://github.com/XXXXXX/YYYYYYYY.git解决方案访问GitHub镜像网站git clone https://github.com.cnpmjs.org/XXXXXX/YYYYYYYY.git缺点：每次都要在github.com后面加.cnpmjs.org缺点弥补方案： 编写alias命令将输入的https://github.com/XXXXXX/YYYYYYYY.git自动替换成git
2021-06-19 16:22:33
203
原创 关于Import pykeops时出现warnning No cuda detect
具体问题在pykeops.common.gpu_utils的函数get_gpu_number()中 libnames = ("libcuda.so", "libcuda.dylib", "cuda.dll") for libname in libnames: try: cuda = ctypes.CDLL(libname) except OSError: continue else:
2021-03-27 19:13:21
812
原创 本地打开服务器上的TensorBoard
假设服务器ip为123.456.789.123，服务器上tensorboard记录文件地址在/home/xxx/code/projectA/logs/tensorboard服务器端tensorboard --logdir=/home/xxx/code/projectA/logs/tensorboard --host=123.456.789.123输入后得到结果TensorBoard x.x.x at https://123.456.789.123:6006/(Press CTRL+C to qui
2021-03-12 15:07:13
299
2
原创 Ubuntu离线安装curl
学校服务器联网要curl，但是服务器curl没装，没网又没法apt在线安装curl（首选）sudo apt install curl离线安装curl从这里下载curl源码curl-7.75.0.tar.gz解压./configure --prefix=/home/xxx/software/curl-7.75.0make -j8make installvim ~/.profile在.profile中添加PATH=/home/xxx/software/curl-7.75.0/bin:{PA
2021-03-08 22:23:32
5093
原创 UBUNTU无ROOT权限解决“Could not resolve hostname XXXXX: Name or service not known“问题
最近在服务器上用torchpack时遇到Could not resolve hostname XXXXX: Name or service not known问题。网上大部分解决方法是将XXXXX添加到/etc/hosts里，但是我没有ROOT权限改不了。通过查阅资料，我发现这个问题大部分是由于使用了系统内有多个版本的openmpi，然后使用了旧版的openmpi。通过使用mpirun --version我发现系统默认使用的openmpi的版本是1.10.2，这个openmpi在/usr/local/bi
2021-01-10 17:25:50
6994
5
原创 python memoryview 简要理解
最近要用到sklearn.neighbor.KDTree，它有一个attribute是data是指构造KDTree数据的memoryview。简单使用后发现memoryview有点像C里的指针import numpy as npfrom sklearn.neighbors import KDTreefrom time import timeif __name__ == '__main__': pts = np.random.randn(10, 3) print("pts:", pt
2020-12-26 15:58:00
456
2
原创 Makefile相关
如何忽略某个文件夹下的指定文件参考链接FILES := $(wildcard $(PATH)/src/*.cc) # 获取文件夹下的所有文件SRC_FILES += $(filter-out $(PATH)/src/a.cc $(PATH)/src/b.cc, $(FILES)) # 使用filter-out过滤a.cc/b.cc如何包含指定头文件# 相对路径是以makefile所在文件夹为起点的-I "$(include_dir_path)/a.h" # 包含include_dir_path
2020-11-17 16:30:06
94
翻译 安装maskrcnn-banchmark时遇到的“AT_CHECK“ is undefined错误
解决方案来自这里。运行环境Ubuntu 18.04Python 3.7、Pytorch 1.8(Nightly)CUDA 10.1错误maskrcnn_benchmark/csrc/cuda/deform_pool_cuda.cu(42): error: identifier "AT_CHECK" is undefinedmaskrcnn_benchmark/csrc/cuda/deform_pool_cuda.cu(68): error: identifier "AT_CHECK" is
2020-10-19 21:27:03
3755
7
原创 自信息、信息熵、互信息
一、自信息在信息论中，是指与概率空间中的单一事件或者离散随机变量的值相关的信息量的量度。简而言之，就是一个事件发生时传达出的信息量。
2020-10-16 00:17:45
2354
原创 从交叉熵（CrossEntropy）到交叉熵损失函数（CrossEntropy Loss）
从交叉熵（CrossEntropy）到交叉熵损失函数（CrossEntropy Loss）所谓交叉熵，这篇文章《什么是交叉熵》讲的很明白。设X是预测分布，Y是真实分布，交叉熵H(X, Y)的表达式为：H(X,Y)=−∑iY[i]logX[i] H(X,Y)=-\sum_i Y[i]\log X[i] H(X,Y)=−i∑Y[i]logX[i]其中logloglog表示自然对数，以eee为底数，不是10为底数。再看到Pytorch Document中对交叉熵损失函数的描述为：loss(X,cla
2020-09-30 17:27:17
288
原创 Git基本使用方法
Git使用方法读研了还不会用git，实属不方便。简单记录一下。Git仓库结构工作目录 <-> 暂存区 <-> 本地仓库 <-> 远程仓库在工作目录不在暂存区的文件为未跟踪的文件(untracked files)，在暂存区不在本地仓库的文件为未staged文件(unstaged files)，在本地仓库不在远程仓库的文件为未提交文件(uncommited files)工作流程工作目录->暂存区git add [ param] < filename&
2020-08-24 16:47:08
167
原创 np.indices函数使用方法
np.indices函数使用方法官方文档中说明，np.indices函数的作用是返回一个代表网格中所有序号的矩阵。具体而言，import numpy as np# 定义矩阵A和BA = np.random.randint(0,100,(2,3))B = np.random.randint(0,100,(2,3,4))# A.shape为(2,3)是一个二维矩阵,因此idx_A.shape为(2,2,3),因为A中每个元素需要2个序号指示位置#同理B.shape为(2,3,4)是一个三维矩阵,
2020-07-27 21:00:29
6476
原创 LPD-Net学习笔记
LPD-Net学习笔记LPD-Net指 LPD-Net: 3D Point Cloud Learning for Large-Scale Place Recognition and Environment Analysis 文中所指LPD-Net。本文主要依据网络结构图简单解释每个模块的作用。Adaptive Local Feature Extraction文中提到大尺度场景点云中每个点局部分布是不均匀的（可以理解为有的地方点密集、有的地方点稀疏）因此在使用KNN构建邻域图时需要自适应的选择K值，
2020-06-02 15:25:20
1763
1
原创 Pytorch中nn.Conv2d数据计算模拟
Pytorch中nn.Conv2d数据计算模拟最近在研究dgcnn网络的源码，其网络架构部分使用的是nn.Conv2d模块。在Pytorch的官方文档中，nn.Conv2d的输入数据为(B, Cin, W, H) 其中B为batch_size表示batch的大小，Cin为输入数据的特征大小（通道数），W、H对于图像数据来说分别表示图像数据的宽和高。输出数据为(B, Cout, W', H')其中...
2020-04-03 23:02:15
927
原创 POJ 1088 滑雪
POJ 1088 滑雪描述Michael喜欢滑雪百这并不奇怪， 因为滑雪的确很刺激。可是为了获得速度，滑的区域必须向下倾斜，而且当你滑到坡底，你不得不再次走上坡或者等待升降机来载你。Michael想知道载一个区域中最长的滑坡。区域由一个二维数组给出。数组的每个数字代表点的高度。下面是一个例子 1 2 3 4 516 17 18 19 615 24 25 20 714 23 22 ...
2019-03-17 17:18:02
138
原创 POJ 1083 Moving Tables
POJ 1083 Moving TablesThe famous ACM (Advanced Computer Maker) Company has rented a floor of a building whose shape is in the following figure.The floor has 200 rooms each on the north side and sout...
2019-03-17 16:27:49
187
原创 POJ 1050 To the Max
POJ 1050 To the Max描述Given a two-dimensional array of positive and negative integers, a sub-rectangle is any contiguous sub-array of size 1*1 or greater located within the whole array. The sum of a ...
2019-03-16 20:57:11
238
原创 POJ 1018 Communication System
1018 Communication System描述We have received an order from Pizoor Communications Inc. for a special communication system. The system consists of several devices. For each device, we are free to choos...
2019-03-16 16:49:57
165
原创 PAT (Advanced Level) 1033 To Fill or Not to Fill （25 分)
1033 To Fill or Not to Fill （25 分)With highways available, driving a car from Hangzhou to any other city is easy. But since the tank capacity of a car is limited, we have to find gas stations on the ...
2019-03-15 19:04:30
236
原创 PAT (Advanced Level) 1035 Password （20 分）
1035 Password （20 分）To prepare for PAT, the judge sometimes has to generate random passwords for the users. The problem is that there are always some confusing passwords since it is hard to distingui...
2019-03-09 19:41:54
155
原创 2019年3月2日PAT后记
2019年3月2日PAT后记昨天去浙大紫金港考了pat，结果很不理想，只有60分，但这是因为，我最后一题快打完的时候，电脑蓝屏了，代码没保存，所以打了1个多小时的代码全没了。蓝屏重启后还剩30分钟，但那时心态有点炸，就没再继续考试了。这里写下，关于紫金港机房编译环境，后序复习方法的总结，希望对后续的考试有所帮助。编译环境在考试前我一直想找紫金港机房有什么编译环境，但是一直找不到，后来我去紫金...
2019-03-03 17:00:36
873
原创 PAT (Advanced Level) 1070 Mooncake （25 分）
1070 Mooncake （25 分）Mooncake is a Chinese bakery product traditionally eaten during the Mid-Autumn Festival. Many types of fillings and crusts can be found in traditional mooncakes according to the r...
2019-02-24 11:07:07
161
原创 PAT (Advanced Level) 1117 Eddington Number （25 分）
1117 Eddington Number （25 分）British astronomer Eddington liked to ride a bike. It is said that in order to show off his skill, he has even defined an “Eddington number”, E – that is, the maximum inte...
2019-02-23 23:16:07
161
原创 PAT (Advanced Level) 1144 The Missing Number （20 分）
1144 The Missing Number （20 分）Given N integers, you are supposed to find the smallest positive integer that is NOT in the given list.Input Specification:Each input file contains one test case. For ...
2019-02-23 22:29:43
141
原创 PAT (Advanced Level) 1145 Hashing - Average Search Time （25 分）
1145 Hashing - Average Search Time （25 分）The task of this problem is simple: insert a sequence of distinct positive integers into a hash table first. Then try to find another sequence of integer keys...
2019-02-23 22:00:57
140
原创 PAT (Advanced Level) 1147 Heaps （30 分）
1147 Heaps （30 分）In computer science, a heap is a specialized tree-based data structure that satisfies the heap property: if P is a parent node of C, then the key (the value) of P is either greater t...
2019-02-23 21:19:40
134
原创 PAT (Advanced Level) 1151 LCA in a Binary Tree （30 分）
1151 LCA in a Binary Tree （30 分）The lowest common ancestor (LCA) of two nodes U and V in a tree is the deepest node that has both U and V as descendants.Given any two nodes in a binary tree, you are...
2019-02-23 20:38:34
147
原创 PAT (Advanced Level) 1150 Travelling Salesman Problem （25 分）
1150 Travelling Salesman Problem （25 分）The “travelling salesman problem” asks the following question: “Given a list of cities and the distances between each pair of cities, what is the shortest possi...
2019-02-23 20:34:56
157
原创 PAT (Advanced Level) 1149 Dangerous Goods Packaging （25 分）
1149 Dangerous Goods Packaging （25 分）When shipping goods with containers, we have to be careful not to pack some incompatible goods into the same container, or we might get ourselves in serious troub...
2019-02-23 20:31:38
120
原创 PAT (Advanced Level) 1148 Werewolf - Simple Version （20 分）
1148 Werewolf - Simple Version （20 分）Werewolf（狼人杀） is a game in which the players are partitioned into two parties: the werewolves and the human beings. Suppose that in a game, player #1 said: "P...
2019-02-23 20:28:48
159
原创 PAT (Advanced Level) 1105 Spiral Matrix （25 分）
1105 Spiral Matrix （25 分）This time your job is to fill a sequence of N positive integers into a spiral matrix in non-increasing order. A spiral matrix is filled in from the first element at the upper...
2019-02-23 12:34:37
123
原创 PAT (Advanced Level) 1100 Mars Numbers （20 分）
1100 Mars Numbers （20 分）People on Mars count their numbers with base 13:Zero on Earth is called “tret” on Mars.The numbers 1 to 12 on Earth is called “jan, feb, mar, apr, may, jun, jly, aug, sep, ...
2019-02-23 11:03:30
163
原创 PAT (Advanced Level) 1099 Build A Binary Search Tree （30 分）
1099 Build A Binary Search Tree （30 分）A Binary Search Tree (BST) is recursively defined as a binary tree which has the following properties:The left subtree of a node contains only nodes with keys ...
2019-02-22 13:30:00
219
原创 PAT (Advanced Level) 1092 To Buy or Not to Buy （20 分）
1092 To Buy or Not to Buy （20 分）Eva would like to make a string of beads with her favorite colors so she went to a small shop to buy some beads. There were many colorful strings of beads. However the...
2019-02-22 12:38:02
241
原创 PAT (Advanced Level) 1096 Consecutive Factors （20 分）
1096 Consecutive Factors （20 分）Among all the factors of a positive integer N, there may exist several consecutive numbers. For example, 630 can be factored as 3×5×6×7, where 5, 6, and 7 are the three...
2019-02-22 11:36:37
212
原创 PAT (Advanced Level) 1048 Find Coins （25 分）
1048 Find Coins （25 分）Eva loves to collect coins from all over the universe, including some other planets like Mars. One day she visited a universal shopping mall which could accept all kinds of coin...
2019-02-21 22:24:20
123
原创 PAT (Advanced Level) 1146 Topological Order （25 分）
1146 Topological Order （25 分）This is a problem given in the Graduate Entrance Exam in 2018: Which of the following is NOT a topological order obtained from the given directed graph? Now you are suppo...
2019-02-21 11:30:40
289
原创 PAT (Advanced Level) 1111 Online Map （30 分）
1111 Online Map （30 分）Input our current position and a destination, an online map can recommend several paths. Now your job is to recommend two paths to your user: one is the shortest, and the other ...
2019-02-20 22:18:18
136
空空如也
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人