Refactoring

.github/workflows/go.yml

@@ -0,0 +1,28 @@
+# This workflow will build a golang project
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go
+
+name: Go
+
+on:
+  push:
+    branches: [ "master" ]
+  pull_request:
+    branches: [ "master" ]
+
+jobs:
+
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Go
+      uses: actions/setup-go@v4
+      with:
+        go-version: '1.22'
+
+    - name: Build
+      run: go build -v ./...
+
+    - name: Test
+      run: go test -v ./...

README.md

@@ -1,94 +1,130 @@
-# Simple photo cataloguer
+# Effortless Photo Organizer
 
-Just copy/hardlink photos (or video, or any other files) from one place to
+[![Go](https://github.com/derfenix/photocatalog/actions/workflows/go.yml/badge.svg)](https://github.com/derfenix/photocatalog/actions/workflows/go.yml)
-another, separating them in sub-directories like `$ROOT/year/month/day/`.
 
-### TL;DR 
+A simple tool to organize your photos, videos, or other files by copying or hardlinking them into a date-based directory structure like `$ROOT/year/month/day/`.
 
-I have a smartphone, I have a Syncthing ~~uugh... SmartThing~~ and all photos
+## TL;DR
-from smartphone nicely synced to my PC without my attention. But I can't just
-keep all photos in synced folder: if I'll clean my phone memory - all photos 
-from pc will be cleaned too. I need to not forget copy files in another 
-place before cleaning phone's memory. Also, I can't  just drop all photos in 
-one dir - I will not find anything there later, and a folder with thousands 
-photos looks like a bad idea from either side.    
-So I create this tool in one evening. All it does - copy (or create hardlinks for)
-files from one place to another, creating basic date-aware directories
-structure for that files.
 
+I use a smartphone and Syncthing to automatically sync my photos to my PC. However, if I clean up my phone's memory, the synced photos on my PC are deleted as well.
+Dumping everything into one folder wasn't an option — finding anything later would be a nightmare. 
+
+To avoid this, I needed a solution to back up and organize my photos without manual effort. So, I built this tool in one evening to solve the problem. It has worked flawlessly for me and might help you too. If you encounter any issues, feel free to open a ticket — I'll do my best to assist.
+
+## Installation
+
+Install the tool via `go`:
 
-## Installing
 ```bash
-go install github.com/derfenix/photocatalog@latest
+go install github.com/derfenix/photocatalog/v2@latest
 ```
-Optionally you could copy created binary from the GO's bin path to 
+
-system or user $PATH, e.g. /usr/local/bin/.
+Optionally, copy the binary to a directory in your system or user's `$PATH` (e.g., `/usr/local/bin`):
+
 ```bash
 sudo cp ${GOPATH}/bin/photocatalog /usr/local/bin/photocatalog
 ```
 
-## Supported formats
+## Organization Modes
-At this moment supported jpeg files with filled exif data or any other 
-files but with names matching pattern `yyymmdd_HHMMSS.ext`. Such 
-names format applied by android's camera software (I guess all cams 
-use this format, fix me if I'm wrong).
 
-There is no support for changing names format without modifying  source code 
+The tool supports the following organization modes:
-at this time.
+
+- **copy** — Copies files to the target directory. If the filesystem supports it, uses Copy-on-Write (COW) for efficiency (via FICLONE ioctl call).
+- **hardlink** — Creates hardlinks to the source files, saving disk space. Ideal (and usable only) if the source and target are on the same partition, 
+though file permissions remain linked to the original. Fallback to copy on fail.
+- **move** — Moves files from the source to the target directory.
+- **symlink** — Creates symbolic links at the target pointing to the source files.
+
+## Supported Formats
+
+- **JPEG and TIFF files** with valid EXIF metadata.
+- Files named in the format `yyyymmdd_HHMMSS.ext` (optionally with suffixes after the timestamp) (e.g., `20230101_123456.jpg`). This format is common in Android cameras and other devices.
+
+If a file lacks EXIF data, the tool falls back to parsing the filename.
+
+Currently, the timestamp format is not customizable. Let me know if support for additional formats is required.
 
 ## Usage
-### One-shot 
+
-#### Copy files (make a COW if fs supports it)
+Arguments
-```bash
+```
-photocalog -mode copy -target ./photos/ ./sync/photos/*
+  -dir-mode string
+        Mode bits for directories can be created while syncing (default "0777")
+  -file-mode string
+        Mode bits for files created while syncing (not applicable for hardlink mode) (default "0644")
+  -mode string
+        Organazing mode (default "hardlink")
+  -overwrite
+        Overwrite existing files
+  -skip-full-sync
+        Skip full sync at startup
+  -source string
+        Source directory
+  -target string
+        Target directory
+  -watch
+        Watch for changes in the source directory (default true)
+
 ```
 
-#### Create hardlinks (only withing one disk partition)
+`-skip-full-sync` and `-watch` are not compatible.
-```bash
+
-photocalog -mode hardlink -target ./photos/ ./sync/photos/*
+`-source` and `-target` are required.
-```
+
-or 
+
-```bash
+## Examples
-photocalog -target ./photos/ ./sync/photos/*
+
+### One-Time Run
+
+#### Copy Files
+```shell
+photocatalog -mode copy -target ./photos/ -source ./sync/photos/
 ```
 
-### Monitor
+#### Create Hardlinks
-#### Copy files (make a COW if fs supports it)
+```shell
-```bash
+photocatalog -mode hardlink -target ./photos/ -source ./sync/photos/
-photocalog -mode copy -target ./photos -monitor ./sync/photos/*
 ```
 
-#### Create hardlinks (only withing one disk partition)
+### Watch Mode
-```bash
+
-photocalog -mode hardlink -target ./photos/ -monitor ./sync/photos/
+Enable continuous monitoring of a source directory:
-```
+
-or 
+#### Copy Files
-```bash
+```shell
-photocalog -target ./photos/ -monitor ./sync/photos/
+photocatalog -mode copy -target ./photos -watch -source ./sync/photos/
 ```
 
-## Install and run monitor service
+#### Create Hardlinks
+```shell
+photocatalog -mode hardlink -target ./photos/ -watch -source ./sync/photos/
+```
 
-### Systemd
+## Running as a Service
-```bash
+
+### Systemd Setup
+
+Install and configure the service:
+```shell
 sh ./init/install_service.sh systemd
 ```
-This command will install unit file, create stub for its config and open
-editor to allow you edit configuration. Config file stored at 
-`$HOME/.config/photocatalog`.
 
-Then enable and start service
+This will:
-```bash
+
+1. Install a systemd unit file.
+2. Create a configuration stub at `$HOME/.config/photocatalog`.
+3. Open the config file for editing.
+
+Enable and start the service:
+```shell
 systemctl --user enable --now photocatalog
 ```
-That's all. Now, if any file will be placed in directory, specified as `MONITOR`
+
-in config file, this file will be copied or hardlinked into the target dir
+Now, files added to the monitored directory (`MONITOR` in the config) will automatically be organized into the target directory under the corresponding subdirectories.
-under corresponding sub-dir. 
 
 ## FAQ
 
-### Why this tool was created if there is awesome XXX tool?
+### Why did you create this tool when awesome tool XXX already exists?
-I had two good reasons:
+Two reasons:
-1. I wanted
+1. I wanted to.
-2. I can
+2. I could.
-

application/application.go

@@ -43,22 +43,24 @@ func (a *Application) Start(ctx context.Context, wg *sync.WaitGroup) error {
 		WithDirMode(os.FileMode(a.config.DirMode)).
 		WithFileMode(os.FileMode(a.config.FileMode)).
 		WithErrLogger(func(err error) {
-			log.Println(err)
+			log.Println("ERROR:", err.Error())
 		})
 
 	if a.config.Overwrite {
 		org = org.WithOverwrite()
 	}
 
-	if err := org.FullSync(ctx); err != nil {
-		return fmt.Errorf("full sync: %w", err)
-	}
-
 	if a.config.Watch {
 		if err := org.Watch(ctx, wg); err != nil {
 			return fmt.Errorf("initialize watch: %w", err)
 		}
 	}
 
+	if !a.config.SkipFullSync {
+		if err := org.FullSync(ctx); err != nil {
+			return fmt.Errorf("full sync: %w", err)
+		}
+	}
+
 	return nil
 }

application/config.go

@@ -14,6 +14,8 @@ const (
 	ModeMove     Mode = "move"
 )
 
+var SupportedModes = []Mode{ModeHardlink, ModeSymlink, ModeMove, ModeCopy}
+
 type Config struct {
 	SourceDir    string
 	TargetDir    string
@@ -22,6 +24,7 @@ type Config struct {
 	DirMode      uint64
 	FileMode     uint64
 	Watch        bool
+	SkipFullSync bool
 }
 
 func (c *Config) Validate() error {
@@ -33,8 +36,12 @@ func (c *Config) Validate() error {
 		return fmt.Errorf("target dir is required")
 	}
 
-	if !slices.Contains([]Mode{ModeHardlink, ModeSymlink, ModeMove, ModeSymlink}, c.Mode) {
+	if !slices.Contains(SupportedModes, c.Mode) {
-		return fmt.Errorf("invalid mode %s", c.Mode)
+		return fmt.Errorf("invalid mode %s, supported modes: %s", c.Mode, SupportedModes)
+	}
+
+	if c.SkipFullSync && !c.Watch {
+		return fmt.Errorf("skip full sync and watch disabled — nothing to do")
 	}
 
 	return nil

init/systemd/photocatalog.service

@@ -7,5 +7,5 @@ WantedBy=default.target
 [Service]
 Type=simple
 EnvironmentFile=/home/%u/.config/photocatalog
-ExecStart=photocatalog -mode $MODE -target $TARGET -monitor $MONITOR -update_mtime $UPDATECTIME
+ExecStart=photocatalog -mode $MODE -target $TARGET -watch -source $MONITOR -skip-full-sync
-ExecStartPre=photocatalog -mode $MODE -target $TARGET ${MONITOR}
+ExecStartPre=photocatalog -mode $MODE -target $TARGET -source ${MONITOR}

internal/organizer/modes/copy.go

@@ -5,12 +5,18 @@ import (
 	"io"
 	"log"
 	"os"
+	"runtime"
+	"sync/atomic"
+
+	"golang.org/x/sys/unix"
 )
 
+var cowDisabled = atomic.Bool{}
+
 type Copy struct{}
 
 func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
-	targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_RDWR|os.O_CREATE, mode)
+	targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_WRONLY|os.O_CREATE, mode)
 	if err != nil {
 		return fmt.Errorf("open target file: %w", err)
 	}
@@ -28,6 +34,17 @@ func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
 		_ = sourceFile.Close()
 	}()
 
+	// Try to do a COW.
+	if runtime.GOOS == "linux" && !cowDisabled.Load() {
+		if err := unix.IoctlFileClone(int(targetFile.Fd()), int(sourceFile.Fd())); err == nil {
+			return nil
+		} else {
+			log.Println(fmt.Errorf("COW attempt for %s failed: %w", targetPath, err))
+			log.Println("Disabling COW until restart")
+			cowDisabled.Store(true)
+		}
+	}
+
 	copySize, err := io.Copy(targetFile, sourceFile)
 	if err != nil {
 		return fmt.Errorf("copy source file: %w", err)

internal/organizer/modes/hardlink.go

@@ -1,21 +1,38 @@
 package modes
 
 import (
-	"fmt"
+	"log"
 	"os"
+	"sync/atomic"
 )
 
+var hardLinkNotSupported = atomic.Bool{}
+
 type HardLink struct {
 }
 
 func (h HardLink) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
+	if hardLinkNotSupported.Load() {
+		return h.fallBack(sourcePath, targetPath, mode)
+	}
+
 	if err := os.Link(sourcePath, targetPath); err != nil {
 		if os.IsExist(err) {
 			return nil
 		}
 
-		return fmt.Errorf("create hard link: %w", err)
+		log.Println("Create hardlink failed:", err.Error())
+		hardLinkNotSupported.Store(true)
+
+		return h.fallBack(sourcePath, targetPath, mode)
 	}
 
 	return nil
 }
+
+func (h HardLink) fallBack(sourcePath string, targetPath string, mode os.FileMode) error {
+	if copyErr := (Copy{}).PlaceIt(sourcePath, targetPath, mode); copyErr != nil {
+		return copyErr
+	}
+	return nil
+}

internal/organizer/organizer.go

@@ -11,12 +11,18 @@ import (
 	"strconv"
 	"strings"
 	"sync"
+	"syscall"
 
 	"github.com/fsnotify/fsnotify"
 
 	"github.com/derfenix/photocatalog/v2/internal/metadata"
 )
 
+const (
+	defaultDirMode  = 0o777
+	defaultFileMode = 0o644
+)
+
 type MetaExtractor interface {
 	Extract(_ string, data io.Reader) (metadata.Metadata, error)
 }
@@ -31,8 +37,8 @@ func NewOrganizer(mode Mode, source, target string) *Organizer {
 		sourceDir: source,
 		targetDir: target,
 
-		dirMode:  0777,
+		dirMode:  defaultDirMode,
-		fileMode: 0644,
+		fileMode: defaultFileMode,
 
 		metaExtractors: map[string]MetaExtractor{
 			"":     &metadata.Default{},
@@ -125,6 +131,8 @@ func (o *Organizer) Watch(ctx context.Context, wg *sync.WaitGroup) error {
 		if err := watcher.Close(); err != nil {
 			o.logErr(fmt.Errorf("close watcher: %w", err))
 		}
+
+		syscall.Sync()
 	}()
 
 	go func() {
@@ -144,15 +152,17 @@ func (o *Organizer) Watch(ctx context.Context, wg *sync.WaitGroup) error {
 					// Add new directories to the watcher.
 					if stat.IsDir() {
 						if err := watcher.Add(event.Name); err != nil {
-							o.logErr(fmt.Errorf("watch dir: %w", err))
+							o.logErr(fmt.Errorf("add the directory %s to watcher: %w", event.Name, err))
 						}
 
 						continue
 					}
 
+					go func() {
 						if err := o.processFile(event.Name); err != nil {
 							o.logErr(fmt.Errorf("process file %s: %w", event.Name, err))
 						}
+					}()
 				}
 
 			case <-ctx.Done():
@@ -175,7 +185,9 @@ func (o *Organizer) FullSync(ctx context.Context) error {
 		}
 
 		if err := o.processFile(path); err != nil {
-			return err
+			log.Printf("Process file `%s` failed: %s", path, err.Error())
+
+			return nil
 		}
 
 		return nil
@@ -198,7 +210,7 @@ func (o *Organizer) getMetaForPath(fp string) (metadata.Metadata, error) {
 
 	meta, err := o.getMetadata(fp, file)
 	if err != nil {
-		return metadata.Metadata{}, fmt.Errorf("get metadata: %w", err)
+		return metadata.Metadata{}, fmt.Errorf("get metadatas: %w", err)
 	}
 
 	return meta, nil
@@ -241,7 +253,7 @@ func (o *Organizer) processFile(sourcePath string) error {
 		return fmt.Errorf("build target path: %w", err)
 	}
 
-	if pathExists(targetPath) && !o.overwrite {
+	if o.pathExists(targetPath) && !o.overwrite {
 		return nil
 	}
 
@@ -268,6 +280,7 @@ func (o *Organizer) BuildTargetPath(sourcePath string, meta metadata.Metadata) (
 		o.targetDir,
 		strconv.Itoa(meta.Created.Year()),
 		strconv.Itoa(int(meta.Created.Month())),
+		strconv.Itoa(meta.Created.Day()),
 		sourcePath,
 	)
 
@@ -275,7 +288,7 @@ func (o *Organizer) BuildTargetPath(sourcePath string, meta metadata.Metadata) (
 }
 
 func (o *Organizer) ensureTargetPath(targetPath string) error {
-	if pathExists(targetPath) {
+	if o.pathExists(targetPath) {
 		return nil
 	}
 
@@ -297,13 +310,15 @@ func (o *Organizer) ensureTargetPath(targetPath string) error {
 	return nil
 }
 
-func pathExists(path string) bool {
+func (o *Organizer) pathExists(path string) bool {
 	_, err := os.Stat(path)
 	if err != nil {
 		if os.IsNotExist(err) {
 			return false
 		}
 
+		o.logErr(fmt.Errorf("pathExists stat %s: %w", path, err))
+
 		return true
 	}
 

main.go

@@ -3,8 +3,10 @@ package main
 import (
 	"context"
 	"flag"
+	"fmt"
 	"log"
 	"os/signal"
+	"slices"
 	"strconv"
 	"sync"
 	"syscall"
@@ -33,43 +35,54 @@ func main() {
 }
 
 func loadCfg() application.Config {
-	cfg := application.Config{
+	cfg := application.Config{}
-		SourceDir: "",
-		TargetDir: "",
-		Mode:      "",
-		Overwrite: false,
-		DirMode:   0,
-		FileMode:  0,
-		Watch:     false,
-	}
 
 	flag.StringVar(&cfg.SourceDir, "source", "", "Source directory")
 	flag.StringVar(&cfg.TargetDir, "target", "", "Target directory")
 	flag.BoolVar(&cfg.Overwrite, "overwrite", false, "Overwrite existing files")
 	flag.BoolVar(&cfg.Watch, "watch", false, "Watch for changes in the source directory")
+	flag.BoolVar(&cfg.Watch, "monitor", false, "Watch for changes in the source directory") // Legacy option
+	flag.BoolVar(&cfg.SkipFullSync, "skip-full-sync", false, "Skip full sync at startup")
 
-	var dirMode string
+	flag.Func("dir-mode", "Mode bits for directories can be created while syncing", func(s string) error {
-	var fileMode string
+		var err error
-	flag.StringVar(&dirMode, "dirmode", "0777", "Mode bits for directories can be created while syncing")
-	flag.StringVar(&fileMode, "filemode", "0644", "Mode bits for files created while syncing (not applicable for hardlink mode)")
 
-	var mode string
+		cfg.DirMode, err = strconv.ParseUint(s, 8, 32)
-	flag.StringVar(&mode, "mode", "hardlink", "Mode")
+		if err != nil {
+			return err
+		}
+
+		return nil
+	})
+
+	flag.Func("file-mode", "Mode bits for files created while syncing (not applicable for hardlink mode)", func(s string) error {
+		var err error
+
+		cfg.FileMode, err = strconv.ParseUint(s, 8, 32)
+		if err != nil {
+			return err
+		}
+
+		return nil
+	})
+
+	flag.Func("mode", "Organizing mode", func(s string) error {
+		cfg.Mode = application.Mode(s)
+
+		if !slices.Contains(application.SupportedModes, cfg.Mode) {
+			return fmt.Errorf("invalid mode, supported modes: %s", application.SupportedModes)
+		}
+
+		return nil
+	})
 
 	flag.Parse()
 
-	cfg.Mode = application.Mode(mode)
+	// Legacy fallback
+	if cfg.SourceDir == "" {
+		log.Println("Source directory not specified. May be using old systemd unit file.")
 
-	var err error
+		cfg.SourceDir = flag.Arg(0)
-
-	cfg.DirMode, err = strconv.ParseUint(dirMode, 8, 32)
-	if err != nil {
-		cfg.DirMode = 0o777
-	}
-
-	cfg.FileMode, err = strconv.ParseUint(fileMode, 8, 32)
-	if err != nil {
-		cfg.DirMode = 0o644
 	}
 
 	return cfg