File Hosting#

python -m http.server is a tool widely known for the convenience it brings to quickly host your files on either local network or publicly.

1
python -m http.server -help
2
usage: python -m http.server [-h] [--cgi] [-b ADDRESS] [-d DIRECTORY] [-p VERSION] [--tls-cert PATH]
3
                             [--tls-key PATH] [--tls-password-file PATH]
4
                             [port]
5

6
positional arguments:
7
  port                  bind to this port (default: 8000)
8

9
options:
10
  -h, --help            show this help message and exit
11
  --cgi                 run as CGI server
12
  -b, --bind ADDRESS    bind to this address (default: all interfaces)
13
  -d, --directory DIRECTORY
14
                        serve this directory (default: current directory)
15
  -p, --protocol VERSION
16
                        conform to this HTTP version (default: HTTP/1.0)
17
  --tls-cert PATH       path to the TLS certificate chain file
18
  --tls-key PATH        path to the TLS key file
19
  --tls-password-file PATH
20
                        path to the password file for the TLS key

However,

It doesn’t support uploading or other features that I would like to have with a utility file server
It is a python module which I dislike for various reasons
Too many people are using and recommending this tool while there being some better alternatives

Contents below is copied from the README.md at

sigoden

dufs

Waiting for api.github.com...

00K

Waiting...

I think I don’t have to explain further how this is generally better.

Dufs#

Dufs is a distinctive utility file server that supports static serving, uploading, searching, accessing control, webdav…

demo

Features#

Serve static files
Download folder as zip file
Upload files and folders (Drag & Drop)
Create/Edit/Search files
Resumable/partial uploads/downloads
Access control
Support https
Support webdav
Easy to use with curl

Install#

With cargo#

1
cargo install dufs

With docker#

1
docker run -v `pwd`:/data -p 5000:5000 --rm sigoden/dufs /data -A

With Homebrew #

1
brew install dufs

Binaries on macOS, Linux, Windows#

Download from Github Releases, unzip and add dufs to your $PATH.

CLI#

1
Dufs is a distinctive utility file server - https://github.com/sigoden/dufs
2

3
Usage: dufs [OPTIONS] [serve-path]
4

5
Arguments:
6
  [serve-path]  Specific path to serve [default: .]
7

8
Options:
9
  -c, --config <file>        Specify configuration file
10
  -b, --bind <addrs>         Specify bind address or unix socket
11
  -p, --port <port>          Specify port to listen on [default: 5000]
12
      --path-prefix <path>   Specify a path prefix
13
      --hidden <value>       Hide paths from directory listings, e.g. tmp,*.log,*.lock
14
  -a, --auth <rules>         Add auth roles, e.g. user:pass@/dir1:rw,/dir2
15
  -A, --allow-all            Allow all operations
16
      --allow-upload         Allow upload files/folders
17
      --allow-delete         Allow delete files/folders
18
      --allow-search         Allow search files/folders
19
      --allow-symlink        Allow symlink to files/folders outside root directory
20
      --allow-archive        Allow download folders as archive file
21
      --allow-hash           Allow ?hash query to get file sha256 hash
22
      --enable-cors          Enable CORS, sets `Access-Control-Allow-Origin: *`
23
      --render-index         Serve index.html when requesting a directory, returns 404 if not found index.html
24
      --render-try-index     Serve index.html when requesting a directory, returns directory listing if not found index.html
25
      --render-spa           Serve SPA(Single Page Application)
26
      --assets <path>        Set the path to the assets directory for overriding the built-in assets
27
      --log-format <format>  Customize http log format
28
      --log-file <file>      Specify the file to save logs to, other than stdout/stderr
29
      --compress <level>     Set zip compress level [default: low] [possible values: none, low, medium, high]
30
      --completions <shell>  Print shell completion script for <shell> [possible values: bash, elvish, fish, powershell, zsh]
31
      --tls-cert <path>      Path to an SSL/TLS certificate to serve with HTTPS
32
      --tls-key <path>       Path to the SSL/TLS certificate's private key
33
  -h, --help                 Print help
34
  -V, --version              Print version

Examples#

Serve current working directory in read-only mode

1
dufs

Allow all operations like upload/delete/search/create/edit…

1
dufs -A

Only allow upload operation

1
dufs --allow-upload

Serve a specific directory

1
dufs Downloads

Serve a single file

1
dufs linux-distro.iso

Serve a single-page application like react/vue

1
dufs --render-spa

Serve a static website with index.html

1
dufs --render-index

Require username/password

1
dufs -a admin:123@/:rw

Listen on specific host

1
dufs -b 127.0.0.1 -p 80

Listen on unix socket

1
dufs -b /tmp/dufs.socket

Use https

1
dufs --tls-cert my.crt --tls-key my.key

API#

Upload a file

1
curl -T path-to-file http://127.0.0.1:5000/new-path/path-to-file

Download a file

1
curl http://127.0.0.1:5000/path-to-file           # download the file
2
curl http://127.0.0.1:5000/path-to-file?hash      # retrieve the sha256 hash of the file

Download a folder as zip file

1
curl -o path-to-folder.zip http://127.0.0.1:5000/path-to-folder?zip

Delete a file/folder

1
curl -X DELETE http://127.0.0.1:5000/path-to-file-or-folder

Create a directory

1
curl -X MKCOL http://127.0.0.1:5000/path-to-folder

Move the file/folder to the new path

1
curl -X MOVE http://127.0.0.1:5000/path -H "Destination: http://127.0.0.1:5000/new-path"

List/search directory contents

1
curl http://127.0.0.1:5000?q=Dockerfile           # search for files, similar to `find -name Dockerfile`
2
curl http://127.0.0.1:5000?simple                 # output names only, similar to `ls -1`
3
curl http://127.0.0.1:5000?json                   # output paths in json format

With authorization (Both basic or digest auth works)

1
curl http://127.0.0.1:5000/file --user user:pass                 # basic auth
2
curl http://127.0.0.1:5000/file --user user:pass --digest        # digest auth

Resumable downloads

1
curl -C- -o file http://127.0.0.1:5000/file

Resumable uploads

1
upload_offset=$(curl -I -s http://127.0.0.1:5000/file | tr -d '\r' | sed -n 's/content-length: //p')
2
dd skip=$upload_offset if=file status=none ibs=1 | \
3
  curl -X PATCH -H "X-Update-Range: append" --data-binary @- http://127.0.0.1:5000/file

Health checks

1
curl http://127.0.0.1:5000/__dufs__/health

Advanced Topics

Access Control#

Dufs supports account based access control. You can control who can do what on which path with --auth/-a.

1
dufs -a admin:admin@/:rw -a guest:guest@/
2
dufs -a user:pass@/:rw,/dir1 -a @/

Use @ to separate the account and paths. No account means anonymous user.
Use : to separate the username and password of the account.
Use , to separate paths.
Use path suffix :rw/:ro set permissions: read-write/read-only. :ro can be omitted.

-a admin:admin@/:rw: admin has complete permissions for all paths.
-a guest:guest@/: guest has read-only permissions for all paths.
-a user:pass@/:rw,/dir1: user has read-write permissions for /*, has read-only permissions for /dir1/*.
-a @/: All paths is publicly accessible, everyone can view/download it.

Auth permissions are restricted by dufs global permissions. If dufs does not enable upload permissions via --allow-upload, then the account will not have upload permissions even if it is granted read-write(:rw) permissions.

Hashed Password#

DUFS supports the use of sha-512 hashed password.

Create hashed password:

1
$ openssl passwd -6 123456 # or `mkpasswd -m sha-512 123456`
2
$6$tWMB51u6Kb2ui3wd$5gVHP92V9kZcMwQeKTjyTRgySsYJu471Jb1I6iHQ8iZ6s07GgCIO69KcPBRuwPE5tDq05xMAzye0NxVKuJdYs/

Use hashed password:

1
dufs -a 'admin:$6$tWMB51u6Kb2ui3wd$5gVHP92V9kZcMwQeKTjyTRgySsYJu471Jb1I6iHQ8iZ6s07GgCIO69KcPBRuwPE5tDq05xMAzye0NxVKuJdYs/@/:rw'

The hashed password contains $6, which can expand to a variable in some shells, so you have to use single quotes to wrap it.

Two important things for hashed passwords:

Dufs only supports sha-512 hashed passwords, so ensure that the password string always starts with $6$ .
Digest authentication does not function properly with hashed passwords.

Hide Paths#

Dufs supports hiding paths from directory listings via option --hidden <glob>,....

1
dufs --hidden .git,.DS_Store,tmp

The glob used in —hidden only matches file and directory names, not paths. So --hidden dir1/file is invalid.

1
dufs --hidden '.*'                          # hidden dotfiles
2
dufs --hidden '*/'                          # hidden all folders
3
dufs --hidden '*.log,*.lock'                # hidden by exts
4
dufs --hidden '*.log' --hidden '*.lock'

Log Format#

Dufs supports customize http log format with option --log-format.

The log format can use following variables.

variable	description
$remote_addr	client address
$remote_user	user name supplied with authentication
$request	full original request line
$status	response status
$http_	arbitrary request header field. examples: $http_user_agent,$ http_referer

The default log format is '$time_iso8601 $log_level - $remote_addr "$request" $status.

1
2022-08-06T06:59:31+08:00 INFO - 127.0.0.1 "GET /" 200

A json log format is also supported.

1
dufs --log-format '{"time":"$time_local","addr":"$remote_addr","uri":"$request_uri", "method":"$request_method","status":$status}'
2

3
{"time":"2022-08-06T06:59:31+08:00","addr":"127.0.0.1","uri":"/", "method":"GET","status":200}

Disable http log

1
dufs --log-format=''

Log user-agent

1
dufs --log-format '$remote_addr "$request" $status $http_user_agent'

1
2022-08-06T06:53:55+08:00 INFO - 127.0.0.1 "GET /" 200 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36

Log remote-user

1
dufs --log-format '$remote_addr $remote_user "$request" $status' -a /@admin:admin -a /folder1@user1:pass1

1
2022-08-06T07:04:37+08:00 INFO - 127.0.0.1 admin "GET /" 200

Environment variables#

All options can be set using environment variables prefixed with DUFS_.

1
[serve-path]                DUFS_SERVE_PATH="."
2
    --config <file>         DUFS_CONFIG=config.yaml
3
-b, --bind <addrs>          DUFS_BIND=0.0.0.0
4
-p, --port <port>           DUFS_PORT=5000
5
    --path-prefix <path>    DUFS_PATH_PREFIX=/dufs
6
    --hidden <value>        DUFS_HIDDEN=tmp,*.log,*.lock
7
-a, --auth <rules>          DUFS_AUTH="admin:admin@/:rw|@/"
8
-A, --allow-all             DUFS_ALLOW_ALL=true
9
    --allow-upload          DUFS_ALLOW_UPLOAD=true
10
    --allow-delete          DUFS_ALLOW_DELETE=true
11
    --allow-search          DUFS_ALLOW_SEARCH=true
12
    --allow-symlink         DUFS_ALLOW_SYMLINK=true
13
    --allow-archive         DUFS_ALLOW_ARCHIVE=true
14
    --allow-hash            DUFS_ALLOW_HASH=true
15
    --enable-cors           DUFS_ENABLE_CORS=true
16
    --render-index          DUFS_RENDER_INDEX=true
17
    --render-try-index      DUFS_RENDER_TRY_INDEX=true
18
    --render-spa            DUFS_RENDER_SPA=true
19
    --assets <path>         DUFS_ASSETS=./assets
20
    --log-format <format>   DUFS_LOG_FORMAT=""
21
    --log-file <file>       DUFS_LOG_FILE=./dufs.log
22
    --compress <compress>   DUFS_COMPRESS=low
23
    --tls-cert <path>       DUFS_TLS_CERT=cert.pem
24
    --tls-key <path>        DUFS_TLS_KEY=key.pem

Configuration File#

You can specify and use the configuration file by selecting the option --config <path-to-config.yaml>.

The following are the configuration items:

1
serve-path: '.'
2
bind: 0.0.0.0
3
port: 5000
4
path-prefix: /dufs
5
hidden:
6
  - tmp
7
  - '*.log'
8
  - '*.lock'
9
auth:
10
  - admin:admin@/:rw
11
  - user:pass@/src:rw,/share
12
  - '@/'  # According to the YAML spec, quoting is required.
13
allow-all: false
14
allow-upload: true
15
allow-delete: true
16
allow-search: true
17
allow-symlink: true
18
allow-archive: true
19
allow-hash: true
20
enable-cors: true
21
render-index: true
22
render-try-index: true
23
render-spa: true
24
assets: ./assets/
25
log-format: '$remote_addr "$request" $status $http_user_agent'
26
log-file: ./dufs.log
27
compress: low
28
tls-cert: tests/data/cert.pem
29
tls-key: tests/data/key_pkcs1.pem

Customize UI#

Dufs allows users to customize the UI with your own assets.

1
dufs --assets my-assets-dir/

If you only need to make slight adjustments to the current UI, you copy dufs’s assets directory and modify it accordingly. The current UI doesn’t use any frameworks, just plain HTML/JS/CSS. As long as you have some basic knowledge of web development, it shouldn’t be difficult to modify.

Your assets folder must contains a index.html file.

index.html can use the following placeholder variables to retrieve internal data.

__INDEX_DATA__: directory listing data
__ASSETS_PREFIX__: assets url prefix

A customized 404.html page is also supported.

Here are some Third-party customize UI project: