Why does `ls` in Linux and macOS show different owners (uid) for the same file?
I copied some files to HFS+, using macOS, ensuring that it was copied exactly. On macOS these copied files have 501 as owner according to ls -han.
I then plug in the HFS+ usb stick into Ubuntu, and there the files have 1000 as owner according to ls -han. Why?
I then tried copying one of the 501 owned files in Ubuntu (to the same HFS+ volume), ensuring that it was copied exactly using cp -a.
Now macOS ls sees the new file as owned by user 1000…
Really? I don’t understand — what was the point of using cp with the -a option if it doesn’t even preserve the owner’s user id? What did I miss?
Update: To clarify, I think my confusion here stems from that — in my mind — HFS supports Unix file permissions natively and should "just work" with them.
I recently learned that cps preserve=timestamps does not, in fact, preserve time stamps (creation dates are reset). Am I now to believe that its preserve=ownership does not preserve ownership?
From the Linux kernel documentation for the hfsplus module:
Mount options
uid=n, gid=n
Specifies the user/group that owns all files on the filesystem that have uninitialized permissions structures. Default: user/group id of the mounting process.
501 is the default UID of the first regular user on modern macOS.
So, apparently macOS does not initialize "permissions structures" for some files. Also, the Apple Technote #1150 indicates the storage of owner ID has an added wrinkle:
ownerID
The Mac OS X user ID of the owner of the file or folder. Mac OS X versions prior to 10.3 treats user ID 99 as if it was the user ID of the user currently logged in to the console. If no user is logged in to the console, user ID 99 is treated as user ID 0 (root). Mac OS X version 10.3 treats user ID 99 as if it was the user ID of the process making the call (in effect, making it owned by everyone simultaneously). These substitutions happen at run time. The actual user ID on disk is not changed.
and later:
Note:
If the S_IFMT field (upper 4 bits) of the fileMode field is zero, then Mac OS X assumes that the permissions structure is uninitialized, and internally uses default values for all of the fields. The default user and group IDs are 99, but can be changed at the time the volume is mounted. This default ownerID is then subject to substitution as described above.
This means that files created by Mac OS 8 and 9, or any other implementation that sets the permissions fields to zeroes, will behave as if the "ignore ownership" option is enabled for those files, even if "ignore ownership" is disabled for the volume as a whole.
S_IFMT referred here is the highest 4 bits of the 16-bit value that is used to store the Unix-style permission bits: 3x read/write/execute, and the setuid/setgid/sticky bits. A regular file needs those highest 4 bits to be set to a specific non-zero value (S_IFREG) or else the backwards compatibility mechanism described above will kick in.
The structure of the HFS+ filesystem clearly opens up a possibility to sometimes play "fast and loose" with the file ownerships, and your results indicate macOS seems to do exactly that in some situations.
For removable media, it would make a certain kind of sense for macOS to automatically enable the "ignore ownership" option as the system that writes the files to the media might not be the same as the one that will be reading it, and the two systems might have entirely different UID mappings, resulting in inconvenience to the user.
So this might just be macOS trying to be user friendly on removable media, and assuming that the user’s physical possession of the removable media is equivalent to a proof of ownership of the data within.
Ubuntu’s first regular user account is created with UID 1000, and that’s apparently the account you mounted the HFS+ volume to Ubuntu as.
Since files created by Linux keep their UID 1000 into macOS, that indicates Linux will populate the HFS+ "permissions structures" with file owner UIDs, and once macOS reads them, they will work as expected.
The classic POSIX timestamps are:
ctime= time of last status/metadata changemtime= time of last modification of contentsatime= time of last access.
A creation time (crtime, or birth time) is not one of them. A filesystem may or may not support creation times, and the exact semantics of it may vary between filesystem types and Unix-style operating systems.
Some filesystem drivers handle assigning the creation time internally and make it outright impossible to modify the crtime of a file afterwards: in such a filesystem, a file that’s been accidentally deleted and restored from a backup may have their classic ctime and mtime restored, but the creation time will reflect the time of restoration from backup, since the file is now no longer the original, although it might be an exact copy of it.
When you copy a file, you plainly create a new file: the idea of "preserving the creation time" across a copy operation is an oxymoron.
A filesystem on its own can track the creation time of a file, but that is not necessarily the same thing as the creation time of the data within the file. If you want to track the latter, you usually need either a version control system, or a file format that can include a metadata field on data creation time… and all applications using that data format must agree on the semantics of what the "data creation time" means, or else it will become meaningless.