| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| bsd:notes:rsync_nas [2025/10/25 00:00] – [USB Speed] Sean Rhone | bsd:notes:rsync_nas [2025/10/26 02:31] (current) – [Mount ext4] Sean Rhone |
|---|
| * https://unix.stackexchange.com/questions/67539/how-to-rsync-only-new-files | * https://unix.stackexchange.com/questions/67539/how-to-rsync-only-new-files |
| * https://stackoverflow.com/questions/15787676/rsync-how-to-do-a-checksum-on-rsync | * https://stackoverflow.com/questions/15787676/rsync-how-to-do-a-checksum-on-rsync |
| |
| * TODO: UFS | |
| |
| ====== rsync ====== | ====== rsync ====== |
| |
| * ''NAS1'' = 10TB Main | * ''NAS1'' = 10TB Main |
| * No partition table ([[linux:notes:rsync_nas#format]]) | * ''ext4'' |
| | * No partition table ([[linux:notes:rsync_nas#format|mkfs.ext4 notes]]) |
| |
| su - | su - |
| |
| * ''NAS2'' = 4TB Backup | * ''NAS2'' = 4TB Backup |
| | * NTFS |
| | * [[windows:notes:ntfs_format|LargeFRS]] |
| |
| su - | su - |
| ugen0.6: <USB 3.1 Device PM ASMedia Technology Inc.> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (0mA)</code> | ugen0.6: <USB 3.1 Device PM ASMedia Technology Inc.> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (0mA)</code> |
| |
| ====== UFS2 NAS ====== | ====== Wipe Drive ====== |
| |
| * 2025/10/24: Linux ext4 -> FreeBSD UFS2 | su - |
| * 10TB Main | |
| |
| ===== Format ===== | geom disk list |
| |
| * ''superfloppy'' partitionless ((not MBR/GPT)) | dd if='/dev/zero' of='/dev/da0' count='8096' |
| * :!: ''newfs'' takes a few minutes ((2025/10/24: last number is ''19531953984'')) | |
| | ====== ZFS NAS ====== |
| | |
| | * https://docs.freebsd.org/en/books/handbook/zfs/ |
| | * 10TB Main |
| |
| su - | su - |
| geom disk list | geom disk list |
| |
| dd if='/dev/zero' of='/dev/da0' count='8096' | zpool create 'NAS' '/dev/da0' |
| |
| newfs -L 'NAS' -O '2' -n -o 'time' '/dev/da0' | zpool status 'NAS' |
| | |
| | zpool import 'NAS' |
| | |
| | zpool export 'NAS' |
| |
| ====== Disconnect Enclosure ====== | ====== Disconnect Enclosure ====== |
| |
| usbconfig -d '/dev/ugen0.5' 'power_off' | usbconfig -d '/dev/ugen0.5' 'power_off' |
| |
| ===== Reference ===== | |
| |
| <code> | |
| ugen0.4: <USB3.1 Hub VIA Labs, Inc.> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) | |
| ugen0.5: <USB2.0 Hub VIA Labs, Inc.> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA) | |
| ugen0.6: <USB 3.1 Device PM ASMedia Technology Inc.> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (0mA)</code> | |
| |
| ====== Mount ====== | ====== Mount ====== |
| * :!: ''USB 3.1 ASMedia (5.0Gbps)'' needs listed for >USB2 speeds ((caps around 37MB/s at USB2)) | * :!: ''USB 3.1 ASMedia (5.0Gbps)'' needs listed for >USB2 speeds ((caps around 37MB/s at USB2)) |
| |
| usbconfig | usbconfig | grep '<USB 3.1 Device PM ASMedia Technology Inc.>' |
| | |
| | usbconfig | grep 'SUPER (5.0Gbps)' |
| | |
| | usbconfig | grep '<USB 3.1 Device PM ASMedia Technology Inc.> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps)' |
| |
| diskinfo -t 'da0' | diskinfo -t 'da0' |
| | |
| | iostat -c6 |
| |
| ===== NAS1 ===== | ===== NAS1 ===== |
| |
| * ''NAS1'' = 10TB Main | * ''NAS1'' = 10TB Main |
| * UFS2 | |
| |
| su - | su - |
| |
| geom disk list | zpool import 'NAS' |
| |
| mkdir -p '/mnt/NAS1' && mount -o 'autoro,noatime' -t 'ufs' -v '/dev/da0' '/mnt/NAS1' | chown -R 'espionage724':'espionage724' '/NAS' |
| | |
| chown -R 'espionage724':'espionage724' '/mnt/NAS1' | |
| |
| ===== NAS2 ===== | ===== NAS2 ===== |
| mkdir -p '/mnt/NAS2' && ntfs-3g -o 'rw,big_writes,uid=1001,gid=1001' '/dev/da1p2' '/mnt/NAS2' | mkdir -p '/mnt/NAS2' && ntfs-3g -o 'rw,big_writes,uid=1001,gid=1001' '/dev/da1p2' '/mnt/NAS2' |
| |
| ===== 4TB Restore ===== | ====== 4TB Restore ====== |
| |
| * 4TB HDD restore to 10TB NAS drive | * 4TB HDD restore to 10TB NAS drive |
| * Exclude ''Videos'', ''Games'', ''Music'' | * Exclude ''Videos'', ''Games'', ''Music'' |
| |
| <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' --exclude 'Videos' --exclude 'Games' --exclude 'Music' '/mnt/NAS2/' '/mnt/NAS1'</code> | <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' --exclude 'Videos' --exclude 'Games' --exclude 'Music' '/mnt/NAS2/' '/NAS'</code> |
| |
| * Games | ===== Games ===== |
| |
| <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/run/media/espionage724/20250824-NAS/Games/' '/run/media/espionage724/NAS/Games'</code> | **** |
| |
| * Music (:!: ''&&'' with ''Videos'' overnight) | <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/mnt/NAS2/Games/' '/NAS/Games'</code> |
| |
| <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/run/media/espionage724/20250824-NAS/Music/' '/run/media/espionage724/NAS/Music'</code> | ===== Music ===== |
| |
| * Videos (some) | * :!: ''&&'' with ''Videos'' overnight |
| |
| <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/run/media/espionage724/20250824-NAS/Videos/' '/run/media/espionage724/NAS/Videos'</code> | <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/mnt/NAS2/Music/' '/NAS/Music'</code> |
| |
| ===== 2TB USB Restore ===== | ===== Videos ===== |
| | |
| | * :!: Only some (rest below) |
| | |
| | <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/mnt/NAS2/Videos/' '/NAS/Videos'</code> |
| | |
| | ====== 2TB USB Restore ====== |
| |
| * Videos 2 (WD ext) | * Videos 2 (WD ext) |
| |
| <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/run/media/espionage724/688EDD408EDD0808/Dragon Ball DBOX' '/run/media/espionage724/NAS/Videos/Anime'</code> | ===== Mount ===== |
| |
| <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/run/media/espionage724/688EDD408EDD0808/Dragon Ball GT' '/run/media/espionage724/NAS/Videos/Anime'</code> | su - |
| |
| ===== Maintenance ===== | geom disk list |
| |
| ==== Repair ==== | gpart list 'da1' |
| |
| sync && udisksctl unmount --block-device '/dev/sda' | kldload 'fusefs' |
| | |
| | mkdir -p '/mnt/USB2' && ntfs-3g -o 'ro,big_writes,uid=1001,gid=1001' '/dev/da1s1' '/mnt/USB2' |
| | |
| | ===== rsync ===== |
| | |
| | <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/mnt/USB2/Dragon Ball GT' '/NAS/Videos/Anime'</code> |
| | |
| | <code>rsync --recursive --update --ignore-existing --verbose --debug='ALL' '/mnt/USB2/Dragon Ball DBOX' '/NAS/Videos/Anime'</code> |
| | |
| | ====== Maintenance ====== |
| | |
| | ===== ZFS Scrub ===== |
| | |
| | su - |
| |
| sudo e2fsck -D -E 'discard,optimize_extents,inode_count_fullmap,check_encoding,unshare_blocks' -f -v -y '/dev/sda' | zpool import 'NAS' |
| |
| ==== Defrag ==== | zpool scrub 'NAS' |
| |
| * :!: First-time seemingly takes longer than the whole 5TB+ restore | zpool status 'NAS' |
| |
| udisksctl mount --block-device '/dev/sda' | zpool export 'NAS' |
| |
| sudo e4defrag -v '/dev/sda' && sync && udisksctl unmount --block-device '/dev/sda' | exit |
| |