mirror of
https://github.com/derfenix/photocatalog.git
synced 2026-03-12 06:34:57 +03:00
Compare commits
11 Commits
v2.0.0-alp
...
bdd3eee69f
| Author | SHA1 | Date | |
|---|---|---|---|
| bdd3eee69f | |||
| 8e315ff557 | |||
| 3ae91523d8 | |||
| 535c790c39 | |||
| 73aa5c4ac2 | |||
| 077f920986 | |||
| 77443e2369 | |||
| 9acb554dd4 | |||
| e3f8f7b8c8 | |||
| 62ca9f2378 | |||
| f73d612666 |
28
.github/workflows/go.yml
vendored
Normal file
28
.github/workflows/go.yml
vendored
Normal file
@@ -0,0 +1,28 @@
|
||||
# This workflow will build a golang project
|
||||
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go
|
||||
|
||||
name: Go
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ "master" ]
|
||||
pull_request:
|
||||
branches: [ "master" ]
|
||||
|
||||
jobs:
|
||||
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v4
|
||||
with:
|
||||
go-version: '1.22'
|
||||
|
||||
- name: Build
|
||||
run: go build -v ./...
|
||||
|
||||
- name: Test
|
||||
run: go test -v ./...
|
||||
132
README.md
132
README.md
@@ -1,94 +1,100 @@
|
||||
# Simple photo cataloguer
|
||||
# Effortless Photo Organizer
|
||||
|
||||
Just copy/hardlink photos (or video, or any other files) from one place to
|
||||
another, separating them in sub-directories like `$ROOT/year/month/day/`.
|
||||
[](https://github.com/derfenix/photocatalog/actions/workflows/go.yml)
|
||||
|
||||
### TL;DR
|
||||
A simple tool to organize your photos, videos, or other files by copying or hardlinking them into a date-based directory structure like `$ROOT/year/month/day/`.
|
||||
|
||||
I have a smartphone, I have a Syncthing ~~uugh... SmartThing~~ and all photos
|
||||
from smartphone nicely synced to my PC without my attention. But I can't just
|
||||
keep all photos in synced folder: if I'll clean my phone memory - all photos
|
||||
from pc will be cleaned too. I need to not forget copy files in another
|
||||
place before cleaning phone's memory. Also, I can't just drop all photos in
|
||||
one dir - I will not find anything there later, and a folder with thousands
|
||||
photos looks like a bad idea from either side.
|
||||
So I create this tool in one evening. All it does - copy (or create hardlinks for)
|
||||
files from one place to another, creating basic date-aware directories
|
||||
structure for that files.
|
||||
## TL;DR
|
||||
|
||||
I use a smartphone and Syncthing to automatically sync my photos to my PC. However, if I clean up my phone's memory, the synced photos on my PC are deleted as well. To avoid this, I needed a solution to back up and organize my photos without manual effort.
|
||||
|
||||
Dumping everything into one folder wasn't an option — finding anything later would be a nightmare. So, I built this tool in one evening to solve the problem. It has worked flawlessly for me and might help you too. If you encounter any issues, feel free to open a ticket — I'll do my best to assist.
|
||||
|
||||
## Installation
|
||||
|
||||
Install the tool via `go`:
|
||||
|
||||
## Installing
|
||||
```bash
|
||||
go install github.com/derfenix/photocatalog@latest
|
||||
go install github.com/derfenix/photocatalog/v2@latest
|
||||
```
|
||||
Optionally you could copy created binary from the GO's bin path to
|
||||
system or user $PATH, e.g. /usr/local/bin/.
|
||||
|
||||
Optionally, copy the binary to a directory in your system or user's `$PATH` (e.g., `/usr/local/bin`):
|
||||
|
||||
```bash
|
||||
sudo cp ${GOPATH}/bin/photocatalog /usr/local/bin/photocatalog
|
||||
```
|
||||
|
||||
## Supported formats
|
||||
At this moment supported jpeg files with filled exif data or any other
|
||||
files but with names matching pattern `yyymmdd_HHMMSS.ext`. Such
|
||||
names format applied by android's camera software (I guess all cams
|
||||
use this format, fix me if I'm wrong).
|
||||
## Organization Modes
|
||||
|
||||
There is no support for changing names format without modifying source code
|
||||
at this time.
|
||||
The tool supports the following organization modes:
|
||||
|
||||
- **copy** — Copies files to the target directory. If the filesystem supports it, uses Copy-on-Write (COW) for efficiency.
|
||||
- **hardlink** — Creates hardlinks to the source files, saving disk space. Ideal if the source and target are on the same partition, though file permissions remain linked to the original.
|
||||
- **move** — Moves files from the source to the target directory.
|
||||
- **symlink** — Creates symbolic links at the target pointing to the source files.
|
||||
|
||||
## Supported Formats
|
||||
|
||||
- **JPEG and TIFF files** with valid EXIF metadata.
|
||||
- Files named in the format `yyyymmdd_HHMMSS.ext` (optionally with suffixes after the timestamp) (e.g., `20230101_123456.jpg`). This format is common in Android cameras and other devices.
|
||||
|
||||
If a file lacks EXIF data, the tool falls back to parsing the filename.
|
||||
|
||||
Currently, the timestamp format is not customizable. Let me know if support for additional formats is required.
|
||||
|
||||
## Usage
|
||||
### One-shot
|
||||
#### Copy files (make a COW if fs supports it)
|
||||
```bash
|
||||
photocalog -mode copy -target ./photos/ ./sync/photos/*
|
||||
|
||||
### One-Time Run
|
||||
|
||||
#### Copy Files
|
||||
```shell
|
||||
photocatalog -mode copy -target ./photos/ -source ./sync/photos/
|
||||
```
|
||||
|
||||
#### Create hardlinks (only withing one disk partition)
|
||||
```bash
|
||||
photocalog -mode hardlink -target ./photos/ ./sync/photos/*
|
||||
```
|
||||
or
|
||||
```bash
|
||||
photocalog -target ./photos/ ./sync/photos/*
|
||||
#### Create Hardlinks
|
||||
```shell
|
||||
photocatalog -mode hardlink -target ./photos/ -source ./sync/photos/
|
||||
```
|
||||
|
||||
### Monitor
|
||||
#### Copy files (make a COW if fs supports it)
|
||||
```bash
|
||||
photocalog -mode copy -target ./photos -monitor ./sync/photos/*
|
||||
### Watch Mode
|
||||
|
||||
Enable continuous monitoring of a source directory:
|
||||
|
||||
#### Copy Files
|
||||
```shell
|
||||
photocatalog -mode copy -target ./photos -watch -source ./sync/photos/
|
||||
```
|
||||
|
||||
#### Create hardlinks (only withing one disk partition)
|
||||
```bash
|
||||
photocalog -mode hardlink -target ./photos/ -monitor ./sync/photos/
|
||||
```
|
||||
or
|
||||
```bash
|
||||
photocalog -target ./photos/ -monitor ./sync/photos/
|
||||
#### Create Hardlinks
|
||||
```shell
|
||||
photocatalog -mode hardlink -target ./photos/ -watch -source ./sync/photos/
|
||||
```
|
||||
|
||||
## Install and run monitor service
|
||||
## Running as a Service
|
||||
|
||||
### Systemd
|
||||
```bash
|
||||
### Systemd Setup
|
||||
|
||||
Install and configure the service:
|
||||
```shell
|
||||
sh ./init/install_service.sh systemd
|
||||
```
|
||||
This command will install unit file, create stub for its config and open
|
||||
editor to allow you edit configuration. Config file stored at
|
||||
`$HOME/.config/photocatalog`.
|
||||
|
||||
Then enable and start service
|
||||
```bash
|
||||
This will:
|
||||
|
||||
1. Install a systemd unit file.
|
||||
2. Create a configuration stub at `$HOME/.config/photocatalog`.
|
||||
3. Open the config file for editing.
|
||||
|
||||
Enable and start the service:
|
||||
```shell
|
||||
systemctl --user enable --now photocatalog
|
||||
```
|
||||
That's all. Now, if any file will be placed in directory, specified as `MONITOR`
|
||||
in config file, this file will be copied or hardlinked into the target dir
|
||||
under corresponding sub-dir.
|
||||
|
||||
Now, files added to the monitored directory (`MONITOR` in the config) will automatically be organized into the target directory under the corresponding subdirectories.
|
||||
|
||||
## FAQ
|
||||
|
||||
### Why this tool was created if there is awesome XXX tool?
|
||||
I had two good reasons:
|
||||
1. I wanted
|
||||
2. I can
|
||||
|
||||
### Why did you create this tool when awesome tool XXX already exists?
|
||||
Two reasons:
|
||||
1. I wanted to.
|
||||
2. I could.
|
||||
|
||||
@@ -50,8 +50,10 @@ func (a *Application) Start(ctx context.Context, wg *sync.WaitGroup) error {
|
||||
org = org.WithOverwrite()
|
||||
}
|
||||
|
||||
if err := org.FullSync(ctx); err != nil {
|
||||
return fmt.Errorf("full sync: %w", err)
|
||||
if !a.config.SkipFullSync {
|
||||
if err := org.FullSync(ctx); err != nil {
|
||||
return fmt.Errorf("full sync: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
if a.config.Watch {
|
||||
|
||||
@@ -15,13 +15,14 @@ const (
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
SourceDir string
|
||||
TargetDir string
|
||||
Mode Mode
|
||||
Overwrite bool
|
||||
DirMode uint64
|
||||
FileMode uint64
|
||||
Watch bool
|
||||
SourceDir string
|
||||
TargetDir string
|
||||
Mode Mode
|
||||
Overwrite bool
|
||||
DirMode uint64
|
||||
FileMode uint64
|
||||
Watch bool
|
||||
SkipFullSync bool
|
||||
}
|
||||
|
||||
func (c *Config) Validate() error {
|
||||
@@ -33,9 +34,13 @@ func (c *Config) Validate() error {
|
||||
return fmt.Errorf("target dir is required")
|
||||
}
|
||||
|
||||
if !slices.Contains([]Mode{ModeHardlink, ModeSymlink, ModeMove, ModeSymlink}, c.Mode) {
|
||||
if !slices.Contains([]Mode{ModeHardlink, ModeSymlink, ModeMove, ModeCopy}, c.Mode) {
|
||||
return fmt.Errorf("invalid mode %s", c.Mode)
|
||||
}
|
||||
|
||||
if c.SkipFullSync && !c.Watch {
|
||||
return fmt.Errorf("skip full sync and watch disabled — nothing to do")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -5,12 +5,18 @@ import (
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"runtime"
|
||||
"sync/atomic"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
var cowDisabled = atomic.Bool{}
|
||||
|
||||
type Copy struct{}
|
||||
|
||||
func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
|
||||
targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_RDWR|os.O_CREATE, mode)
|
||||
targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_WRONLY|os.O_CREATE, mode)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open target file: %w", err)
|
||||
}
|
||||
@@ -28,6 +34,17 @@ func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
|
||||
_ = sourceFile.Close()
|
||||
}()
|
||||
|
||||
// Try to do a COW.
|
||||
if runtime.GOOS == "linux" && !cowDisabled.Load() {
|
||||
if err := unix.IoctlFileClone(int(targetFile.Fd()), int(sourceFile.Fd())); err == nil {
|
||||
return nil
|
||||
} else {
|
||||
log.Println(fmt.Errorf("COW attempt for %s failed: %w", targetPath, err))
|
||||
log.Println("Disabling COW until restart")
|
||||
cowDisabled.Store(true)
|
||||
}
|
||||
}
|
||||
|
||||
copySize, err := io.Copy(targetFile, sourceFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("copy source file: %w", err)
|
||||
|
||||
@@ -17,6 +17,11 @@ import (
|
||||
"github.com/derfenix/photocatalog/v2/internal/metadata"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultDirMode = 0o777
|
||||
defaultFileMode = 0o644
|
||||
)
|
||||
|
||||
type MetaExtractor interface {
|
||||
Extract(_ string, data io.Reader) (metadata.Metadata, error)
|
||||
}
|
||||
@@ -31,8 +36,8 @@ func NewOrganizer(mode Mode, source, target string) *Organizer {
|
||||
sourceDir: source,
|
||||
targetDir: target,
|
||||
|
||||
dirMode: 0777,
|
||||
fileMode: 0644,
|
||||
dirMode: defaultDirMode,
|
||||
fileMode: defaultFileMode,
|
||||
|
||||
metaExtractors: map[string]MetaExtractor{
|
||||
"": &metadata.Default{},
|
||||
@@ -268,6 +273,7 @@ func (o *Organizer) BuildTargetPath(sourcePath string, meta metadata.Metadata) (
|
||||
o.targetDir,
|
||||
strconv.Itoa(meta.Created.Year()),
|
||||
strconv.Itoa(int(meta.Created.Month())),
|
||||
strconv.Itoa(meta.Created.Day()),
|
||||
sourcePath,
|
||||
)
|
||||
|
||||
|
||||
17
main.go
17
main.go
@@ -33,25 +33,18 @@ func main() {
|
||||
}
|
||||
|
||||
func loadCfg() application.Config {
|
||||
cfg := application.Config{
|
||||
SourceDir: "",
|
||||
TargetDir: "",
|
||||
Mode: "",
|
||||
Overwrite: false,
|
||||
DirMode: 0,
|
||||
FileMode: 0,
|
||||
Watch: false,
|
||||
}
|
||||
cfg := application.Config{}
|
||||
|
||||
flag.StringVar(&cfg.SourceDir, "source", "", "Source directory")
|
||||
flag.StringVar(&cfg.TargetDir, "target", "", "Target directory")
|
||||
flag.BoolVar(&cfg.Overwrite, "overwrite", false, "Overwrite existing files")
|
||||
flag.BoolVar(&cfg.Watch, "watch", false, "Watch for changes in the source directory")
|
||||
flag.BoolVar(&cfg.Watch, "watch", true, "Watch for changes in the source directory")
|
||||
flag.BoolVar(&cfg.SkipFullSync, "skip-full-sync", false, "Skip full sync at startup")
|
||||
|
||||
var dirMode string
|
||||
var fileMode string
|
||||
flag.StringVar(&dirMode, "dirmode", "0777", "Mode bits for directories can be created while syncing")
|
||||
flag.StringVar(&fileMode, "filemode", "0644", "Mode bits for files created while syncing (not applicable for hardlink mode)")
|
||||
flag.StringVar(&dirMode, "dir-mode", "0777", "Mode bits for directories can be created while syncing")
|
||||
flag.StringVar(&fileMode, "file-mode", "0644", "Mode bits for files created while syncing (not applicable for hardlink mode)")
|
||||
|
||||
var mode string
|
||||
flag.StringVar(&mode, "mode", "hardlink", "Mode")
|
||||
|
||||
Reference in New Issue
Block a user