Ygtjki

We are supposed to store our ongoing projects on a rather small (~4TB) data server. Not surprisingly, it is constantly overflowing and people need to move off less recent files manually.

Is there an easy (aka standard command-line) way to find out which users take up the most space in a directory? i.e. summing up the size of all files in a directory and all sub-directories belonging to each user?

Edit: ideally not following symlinks

To find the disk usage by all users on the disk, you can use the following script:

#! /bin/bash

DIRECTORY_TO_SCAN=/home

readarray -t user_list<<<"$(cat /etc/passwd | cut -d ':' -f 1)"

for u in "$user_list[@]" ; do
 printf "Scanning for user: %30s" "$u"
 du -ch $(find "$DIRECTORY_TO_SCAN" -user "$u" 2>/dev/null) 2>/dev/null | sed -e '/total/!d;s/^/ /'
done

This script will scan the list of all users, find the files you can read and they own and compute the total space those files use.

(The part below is kept for the sake of completeness)

If you want to know the full size of a directory, you can use the du command:

du -sh directory1 directory2

will print the size of the directory and all its contents.

To actually solve your problem, you could use user (or group) quotas. This would allow you to get a detailled report of how much space each user/group is taking and set hard limits about how much space they may use.

In situations like this, I usually first use du -kx | sort -rn | less to list the directories in a largest-first order. That tells me where the biggest individual disk hogs are, so I'll know where to look. But as @SimonDoppler said, if you don't have at least r-x access to all the sub-directories, your listing will be incomplete: you'll only be able to get the sizes of directories you'll be able to access.

Remember: normally you only need to have write access to a directory to delete any files in it. Only if the directory has the sticky bit set (i.e. the last letter in the permission letter string is a t instead of x), you need be the owner of the file in order to delete it.

If there is no quota available, you may need to do something like this:

#!/bin/sh
if [ "$1" = "" ] || [ "$1" = "-h" ]
then
 echo "Usage: $0##*/ <directory> <username(s)...>" >&2
 exit 64 # EX_USAGE
fi
if ! [ -d "$1" ]
then
 echo "ERROR: directory $1 does not exist" >&2
 exit 66 # EX_NOINPUT
fi

REPORTROOT="$1"
shift
for U in "$@"
do
 # Find all files under $REPORTROOT owned by a particular user,
 # get their sizes and sum them together.
 DISKUSE=$(find "$REPORTROOT" -type f -user "$U" 
 -exec stat -c %s + 2>/dev/null 
 | awk 's+=$1 END printf("%ldn", s)')
 # Display the result for this user.
 printf "%16s: %dn" "$U" "$DISKUSE"
done

Note that running this may take a while.

The "calculate a sum of a list of numbers" awk one-liner is from this Stack Overflow post. Note the comments of the answer.