0x00 Introduction
Cloud disk providers like baidu, 115, yunfile have provided us with convenient cloud storage services which enables us to gain access to file across multiple platforms and devices.
Since our works are often related to word, excel, ppt and etc, we rely heavily on a desktop or laptop to produce productivity in our routine life, but there’s always sometime we don’t have our computer at hand while we need to work on some editing stuff, the only thing we can have within grasp is our mobile, so here’s the need, we need a service to help us synchronize files between computer and mobile phone, then aforementioned provider emerged, responding to the call.
They did, on one hand, help us throw away OTG cable and u disk, we don’t have to carry a heavy laptop anymore, just an iPad or a mobile phone is enough. But on the other hand, we have to realize that all these files we uploaded to cloud disk is within the scope of supervision, which means that these files are not only accessible to us but also to service’s administrator. What’s worse, our files are exposed to risk of leaking and stealing because there is and will always be vulnerability in our account, once the provider is hacked or other providers or platforms leak the password you are using, these files are not belong only to yourselves anymore, these hackers can download them easily without notifying you.
Thankfully, there’s another way to get around these risks, the answer is quite easy and simple, just build one cloud disk on your own.
0x01 FileBrowser
The File Browser is a web-based file manager. It allows you to perform basic management operations on remote’s files on anytime and anywhere, such as: create, delete, move, copy, and more. In addition to allowing you to manage your files, there are other features. It supports the management of multiple users, and each user can have files and permissions that he or she can access. It also supports file sharing, just like the web disk, you can use it to share files with your friends. You can also use it to execute some Linux commands. For example, if you want to clone a codebase in the current directory, you can use it to execute commands such as git clone/pull/push.
Because this software uses golang as backend and vue as frontend, so it can run on multiple linux branches and windows operating system. Here I will give an example on how to get it running on Linux.
Install
You have only to type in one line code in command line to install File Browser on your computer:
1 | curl -fsSL https://filebrowser.xyz/get.sh | bash |
If you are running a docker, a docker image
is also available:
1 2 3 4 5 6 | docker run \ -v /path/to/root:/srv \ -v /path/filebrowser.db:/database.db \ -v /path/.filebrowser.json:/.filebrowser.json \ -p 80:80 \ filebrowser/filebrowser |
In case that you want to install it manually, here’s its github repo link:
https://github.com/filebrowser/filebrowser
Configure
After installation, hit
1 | filebrowser -r /path/to/your/files |
in command line, your file browser will be up and running.
The default username and password will be admin.
File Browser runs in the foreground by default, if you want to switch it to background, you can use nohup
:
1 | nohup filebrowser -d /etc/filebrowser.db >/dev/null 2>&1 & |
In addition, if you want to kill the process:
1 | kill -9 $(pidof filebrowser) |
If you want to run it automatically after booting:
1 | sed -i '/exit 0/i\nohup filebrowser -d \/etc\/filebrowser.db >\/dev\/null 2>&1 &' /etc/rc.local |
and cancel automatic run:
1 | sed -i '/nohup filebrowser -d \/etc\/filebrowser.db >\/dev\/null 2>&1 &/d' /etc/rc.local |
Actually, there are more available configurations which you can feel free to alter, for instance, if you want to create database file in path you want, just hit the code
1 | filebrowser -d path_to_database_file.db config init |
if you want to change the default listening port, you can do like this
1 | filebrowser -d /etc/filebrowser.db config set --port port_num_you_want |
if you want to define the path to store log
1 | filebrowser -d /etc/filebrowser.db config set --log path_to_your_log.log |
if you want to add another admin user
1 | filebrowser -d /etc/filebrowser.db users add another_username another_password --perm.admin |
For more configuration details, you can visit the official site for documents : https://filebrowser.xyz/
0x02 aria2
Up to now, you have already built a private cloud disk for yourself, but if you want to go further, you can add aria2 to obtain the ability of off-line download.
aria2 is a lightweight multi-protocol & multi-source command-line download utility. It supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink. aria2 can be manipulated via built-in JSON-RPC and XML-RPC interfaces.
In short, similar to Thunder or other BT clients, aria2 is also a BT/PT download client. And with the help of it, you can easily download files and store them on you vps or computer. And combined with File Browser, your private cloud disk will have full capability of download media or files off-line and watch/download them on-line.
Install
If you’re running centos, you should deploy epel source first:
1 | yum -y install epel-release |
then install aria2 via yum
:
1 | yum -y install aria2 |
if you’re running ubuntu, just install aria by apt-get
:
1 | sudo apt-get install aria2 |
Configure
you will need to create configuration file for aria2 before you run it:
1 2 3 | mkdir /etc/.aria2 touch /etc/.aria2/aria2.session vi etc/.aria2/aria2.conf |
after doing this, you can enter or paste following configurations into aria2.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | ## 下载设置 ## # 断点续传 continue=true # 最大同时下载任务数, 运行时可修改, 默认:5 max-concurrent-downloads=5 # 单个任务最大线程数, 添加时可指定, 默认:5 split=16 # 最小文件分片大小, 添加时可指定, 取值范围1M -1024M, 默认:20M # 假定size=10M, 文件为20MiB 则使用两个来源下载; 文件为15MiB 则使用一个来源下载 min-split-size=1M # 同一服务器连接数, 添加时可指定, 默认:1 max-connection-per-server=16 # 断开速度过慢的连接 lowest-speed-limit=0 # 整体下载速度限制, 运行时可修改, 默认:0 #max-overall-download-limit=0 # 单个任务下载速度限制, 默认:0 #max-download-limit=0 # 整体上传速度限制, 运行时可修改, 默认:0 #max-overall-upload-limit=0 # 单个任务上传速度限制, 默认:0 #max-upload-limit=0 # 禁用IPv6, 默认:false #disable-ipv6=true # 当服务器返回503错误时, aria2会尝试重连 # 尝试重连次数, 0代表无限, 默认:5 max-tries=0 # 重连冷却, 默认:0 #retry-wait=0 ## 进度保存相关 ## # 从会话文件中读取下载任务 # 开启该参数后aria2将只接受session中的任务, 这意味着aria2一旦使用conf后将不再接受来自终端的任务, 所以该条只需要在启动rpc时加上就可以了 #input-file=/Users/name/.aria2/aria2.session # 在Aria2退出时保存`错误/未完成`的下载任务到会话文件 save-session=/Users/name/.aria2/aria2.session # 定时保存会话, 0为退出时才保存, 需1.16.1以上版本, 默认:0 save-session-interval=60 # 强制保存会话, 即使任务已经完成, 默认:false # 较新的版本开启后会在任务完成后依然保留.aria2文件 #force-save=false ## RPC相关设置 ## # 启用RPC, 默认:false enable-rpc=true # 允许所有来源, 默认:false rpc-allow-origin-all=true # 允许非外部访问, 默认:false rpc-listen-all=true # 事件轮询方式, 取值:[epoll, kqueue, port, poll, select], 不同系统默认值不同 event-poll=kqueue # RPC监听端口, 端口被占用时可以修改, 默认:6800 #rpc-listen-port=6800 # 设置的RPC授权令牌, v1.18.4新增功能, 取代 --rpc-user 和 --rpc-passwd 选项 rpc-secret=123456 # 设置的RPC访问用户名, 此选项新版已废弃, 建议改用 --rpc-secret 选项 #rpc-user=<USER> # 设置的RPC访问密码, 此选项新版已废弃, 建议改用 --rpc-secret 选项 #rpc-passwd=<PASSWD> # 是否启用 RPC 服务的 SSL/TLS 加密, # 启用加密后 RPC 服务需要使用 https 或者 wss 协议连接 #rpc-secure=true # 在 RPC 服务中启用 SSL/TLS 加密时的证书文件(.pem/.crt) #rpc-certificate=/root/xxx.pem # 在 RPC 服务中启用 SSL/TLS 加密时的私钥文件(.key) #rpc-private-key=/root/xxx.key ## BT/PT下载相关 ## # 当下载的是一个种子(以.torrent结尾)时, 自动开始BT任务, 默认:true #follow-torrent=true # BT监听端口, 当端口被屏蔽时使用, 默认:6881-6999 #listen-port=51413 # 单个种子最大连接数, 默认:55 #bt-max-peers=55 # 打开DHT功能, PT需要禁用, 默认:true #enable-dht=false # 打开IPv6 DHT功能, PT需要禁用, 默认:true #enable-dht6=false # DHT网络监听端口, 默认:6881-6999 #dht-listen-port=6881-6999 # 本地节点查找, PT需要禁用, 默认:false bt-enable-lpd=true # 种子交换, PT需要禁用, 默认:true #enable-peer-exchange=true # 每个种子限速, 对少种的PT很有用, 默认:50K #bt-request-peer-speed-limit=50K # 客户端伪装, PT需要 #peer-id-prefix=-TR2770- #user-agent=Transmission/2.77 # 当种子的分享率达到这个数时, 自动停止做种, 0为一直做种, 默认:1.0 #seed-ratio=0 # BT校验相关, 默认:true #bt-hash-check-seed=true # 继续之前的BT任务时, 无需再次校验, 默认:false bt-seed-unverified=true # 保存磁力链接元数据为种子文件(.torrent文件), 默认:false bt-save-metadata=true # 强制加密, 防迅雷必备 #bt-require-crypto=true ## 磁盘相关 ## #文件保存路径, 默认为当前启动位置 dir=/Users/name/Downloads/aria2 #另一种Linux文件缓存方式, 使用前确保您使用的内核支持此选项, 需要1.15及以上版本(?) enable-mmap=true # 文件预分配方式, 能有效降低磁盘碎片, 默认:prealloc # 预分配所需时间: 快none < trunc < falloc < prealloc慢 # falloc仅仅比trunc慢0.06s # 磁盘碎片: 无falloc = prealloc < trunc = none有 # 推荐优先级: 高falloc --> prealloc --> trunc -->none低 # EXT4, btrfs, xfs, NTFS等新型文件系统建议使用falloc, falloc(fallocate)在这些文件系统上可以瞬间创建完整的空文件 # trunc(ftruncate) 同样是是瞬间创建文件, 但是与falloc的区别是创建出的空文件不占用实际磁盘空间 # prealloc 传统的创建完整的空文件, aria2会一直等待直到分配结束, 也就是说如果是在HHD上下载10G文件,那么你的aria2将会一直等待你的硬盘持续满载工作直到10G文件创建完成后才会开始下载 # none将不会预分配, 磁盘碎片程度受下面的disk-cache影响, trunc too # 请勿在传统文件系统如:EXT3, FAT32上使用falloc, 它的实际效果将与prealloc相同 # MacOS建议使用prealloc, 因为它不支持falloc, 也不支持trunc, but可以尝试用brew安装truncate以支持trunc(ftruncate) # 事实上我有些不能理解trunc在aria2中的角色, 它与none几乎没有区别, 也就是说:太鸡肋了 file-allocation=trunc # 启用磁盘缓存, 0为禁用缓存, 需1.16以上版本, 默认:16M disk-cache=64M |
p.s. in rpc-secret
field you should specify a password, which will be needed for WebGUI administration, and if you want to enable ssl support, uncomment rpc-secure
, rpc-certificate
and rpc-private-key
field, then replace the cert path and key path with your own.
Up to now, it is still inconvenient for us to add off-line download task, if you are not geek like, you may probably need a web GUI for aria2, here I would recommend you using AriaNG.
AriaNg is a modern web frontend making aria2 easier to use. AriaNg is written in pure html & javascript, thus it does not need any compilers or runtime environment. You can just put AriaNg in your web server and open it in your browser. AriaNg uses responsive layout, and supports any desktop or mobile devices.
you can download it from Github: [download]https://github.com/mayswind/AriaNg[/download]
It is pure Html & Javascript, just put files in src to your wwwroot you will have it.
In the setting page, fill in the RPC address and key you set in aria2, then refresh the page, your off-line downloader is running without any fault.
Now, you can enjoy your private cloud disk equipped with off-line download.