mirror of
https://github.com/derfenix/photocatalog.git
synced 2026-03-11 21:35:34 +03:00
Compare commits
9 Commits
v2.0.0-alp
...
v2.0.0-alp
| Author | SHA1 | Date | |
|---|---|---|---|
| 3ae91523d8 | |||
| 535c790c39 | |||
| 73aa5c4ac2 | |||
| 077f920986 | |||
| 77443e2369 | |||
| 9acb554dd4 | |||
| e3f8f7b8c8 | |||
| 62ca9f2378 | |||
| f73d612666 |
28
.github/workflows/go.yml
vendored
Normal file
28
.github/workflows/go.yml
vendored
Normal file
@@ -0,0 +1,28 @@
|
||||
# This workflow will build a golang project
|
||||
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go
|
||||
|
||||
name: Go
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ "master" ]
|
||||
pull_request:
|
||||
branches: [ "master" ]
|
||||
|
||||
jobs:
|
||||
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v4
|
||||
with:
|
||||
go-version: '1.22'
|
||||
|
||||
- name: Build
|
||||
run: go build -v ./...
|
||||
|
||||
- name: Test
|
||||
run: go test -v ./...
|
||||
75
README.md
75
README.md
@@ -1,25 +1,23 @@
|
||||
# Simple photo cataloguer
|
||||
[](https://github.com/derfenix/photocatalog/actions/workflows/go.yml)
|
||||
|
||||
# Effortless Photo Organizer
|
||||
|
||||
Just copy/hardlink photos (or video, or any other files) from one place to
|
||||
another, separating them in sub-directories like `$ROOT/year/month/day/`.
|
||||
|
||||
### TL;DR
|
||||
|
||||
I have a smartphone, I have a Syncthing ~~uugh... SmartThing~~ and all photos
|
||||
from smartphone nicely synced to my PC without my attention. But I can't just
|
||||
keep all photos in synced folder: if I'll clean my phone memory - all photos
|
||||
from pc will be cleaned too. I need to not forget copy files in another
|
||||
place before cleaning phone's memory. Also, I can't just drop all photos in
|
||||
one dir - I will not find anything there later, and a folder with thousands
|
||||
photos looks like a bad idea from either side.
|
||||
So I create this tool in one evening. All it does - copy (or create hardlinks for)
|
||||
files from one place to another, creating basic date-aware directories
|
||||
structure for that files.
|
||||
I use a smartphone along with Syncthing to seamlessly sync all my photos to my PC without any manual effort. However, there's a catch: I can't keep all my photos in the synced folder indefinitely. If I clear my phone's memory, the photos on my PC get deleted as well. To avoid this, I need to remember to copy the files to another location before cleaning up my phone.
|
||||
|
||||
Simply dumping all my photos into one folder isn't a solution either — finding anything later would be a nightmare, and a folder with thousands of unsorted photos is far from ideal.
|
||||
|
||||
To address these issues, I created this tool in just one evening. Its primary purpose is to copy (or create hardlinks for) files from one location to another, while organizing them into a simple, date-based directory structure.
|
||||
|
||||
This tool was built for personal use and has been serving me well for quite some time without any problems. However, if you encounter any issues, feel free to report them — I’d be happy to help.
|
||||
|
||||
## Installing
|
||||
```bash
|
||||
go install github.com/derfenix/photocatalog@latest
|
||||
go install github.com/derfenix/photocatalog/v2@latest
|
||||
```
|
||||
Optionally you could copy created binary from the GO's bin path to
|
||||
system or user $PATH, e.g. /usr/local/bin/.
|
||||
@@ -27,44 +25,63 @@ system or user $PATH, e.g. /usr/local/bin/.
|
||||
sudo cp ${GOPATH}/bin/photocatalog /usr/local/bin/photocatalog
|
||||
```
|
||||
|
||||
## Migrating from v0.*
|
||||
|
||||
TODO
|
||||
|
||||
## Organization modes
|
||||
|
||||
Next organization modes supported:
|
||||
|
||||
- **copy** — copy files to target root. Make COW (using syscall) if FS supports it.
|
||||
- **hardlink** — create hardlink to the source file instead of copying.
|
||||
The best choice if source and target are in same partition for compatibility
|
||||
and resource usage, but we can't chmod target files, because of original file mode will
|
||||
be changed too.
|
||||
- **move** — moves original files to new place.
|
||||
- **symlink** — create a symlink at the target for the source files.
|
||||
|
||||
## Supported formats
|
||||
At this moment supported jpeg files with filled exif data or any other
|
||||
files but with names matching pattern `yyymmdd_HHMMSS.ext`. Such
|
||||
names format applied by android's camera software (I guess all cams
|
||||
At this moment supported jpeg and tiff files with filled exif data and any other
|
||||
files but with names matching pattern `yyymmdd_HHMMSS.ext` with optional suffixes after a timestamp.
|
||||
Such names format applied by the Android's camera software (I guess all cams
|
||||
use this format, fix me if I'm wrong).
|
||||
|
||||
There is no support for changing names format without modifying source code
|
||||
at this time.
|
||||
Jpeg/Tiff files without modification date if exif will be fallen back to the name parsing.
|
||||
|
||||
No able to change names format without modifying source code for now. Just because
|
||||
I have reasons to believe that this format is the most popular for the application use cases.
|
||||
But let me know if you need different timestamp formats support.
|
||||
|
||||
## Usage
|
||||
### One-shot
|
||||
#### Copy files (make a COW if fs supports it)
|
||||
#### Copy files
|
||||
```bash
|
||||
photocalog -mode copy -target ./photos/ ./sync/photos/*
|
||||
photocalog -mode copy -target ./photos/ -source ./sync/photos/
|
||||
```
|
||||
|
||||
#### Create hardlinks (only withing one disk partition)
|
||||
#### Create hardlinks
|
||||
```bash
|
||||
photocalog -mode hardlink -target ./photos/ ./sync/photos/*
|
||||
photocalog -mode hardlink -target ./photos/ -source ./sync/photos/
|
||||
```
|
||||
or
|
||||
```bash
|
||||
photocalog -target ./photos/ ./sync/photos/*
|
||||
photocalog -target ./photos/ -source ./sync/photos/*
|
||||
```
|
||||
|
||||
### Monitor
|
||||
#### Copy files (make a COW if fs supports it)
|
||||
### Watch mode
|
||||
#### Copy files
|
||||
```bash
|
||||
photocalog -mode copy -target ./photos -monitor ./sync/photos/*
|
||||
photocalog -mode copy -target ./photos -watch -source ./sync/photos/
|
||||
```
|
||||
|
||||
#### Create hardlinks (only withing one disk partition)
|
||||
#### Create hardlinks
|
||||
```bash
|
||||
photocalog -mode hardlink -target ./photos/ -monitor ./sync/photos/
|
||||
photocalog -mode hardlink -target ./photos/ -watch -source ./sync/photos/
|
||||
```
|
||||
or
|
||||
```bash
|
||||
photocalog -target ./photos/ -monitor ./sync/photos/
|
||||
photocalog -target ./photos/ -watch -source ./sync/photos/
|
||||
```
|
||||
|
||||
## Install and run monitor service
|
||||
@@ -89,6 +106,6 @@ under corresponding sub-dir.
|
||||
|
||||
### Why this tool was created if there is awesome XXX tool?
|
||||
I had two good reasons:
|
||||
1. I wanted
|
||||
1. I wish
|
||||
2. I can
|
||||
|
||||
|
||||
@@ -50,8 +50,10 @@ func (a *Application) Start(ctx context.Context, wg *sync.WaitGroup) error {
|
||||
org = org.WithOverwrite()
|
||||
}
|
||||
|
||||
if err := org.FullSync(ctx); err != nil {
|
||||
return fmt.Errorf("full sync: %w", err)
|
||||
if !a.config.SkipFullSync {
|
||||
if err := org.FullSync(ctx); err != nil {
|
||||
return fmt.Errorf("full sync: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
if a.config.Watch {
|
||||
|
||||
@@ -15,13 +15,14 @@ const (
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
SourceDir string
|
||||
TargetDir string
|
||||
Mode Mode
|
||||
Overwrite bool
|
||||
DirMode uint64
|
||||
FileMode uint64
|
||||
Watch bool
|
||||
SourceDir string
|
||||
TargetDir string
|
||||
Mode Mode
|
||||
Overwrite bool
|
||||
DirMode uint64
|
||||
FileMode uint64
|
||||
Watch bool
|
||||
SkipFullSync bool
|
||||
}
|
||||
|
||||
func (c *Config) Validate() error {
|
||||
@@ -33,9 +34,13 @@ func (c *Config) Validate() error {
|
||||
return fmt.Errorf("target dir is required")
|
||||
}
|
||||
|
||||
if !slices.Contains([]Mode{ModeHardlink, ModeSymlink, ModeMove, ModeSymlink}, c.Mode) {
|
||||
if !slices.Contains([]Mode{ModeHardlink, ModeSymlink, ModeMove, ModeCopy}, c.Mode) {
|
||||
return fmt.Errorf("invalid mode %s", c.Mode)
|
||||
}
|
||||
|
||||
if c.SkipFullSync && !c.Watch {
|
||||
return fmt.Errorf("skip full sync and watch disabled — nothing to do")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -5,12 +5,18 @@ import (
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"runtime"
|
||||
"sync/atomic"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
var cowDisabled = atomic.Bool{}
|
||||
|
||||
type Copy struct{}
|
||||
|
||||
func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
|
||||
targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_RDWR|os.O_CREATE, mode)
|
||||
targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_WRONLY|os.O_CREATE, mode)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open target file: %w", err)
|
||||
}
|
||||
@@ -28,6 +34,17 @@ func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
|
||||
_ = sourceFile.Close()
|
||||
}()
|
||||
|
||||
// Try to do a COW.
|
||||
if runtime.GOOS == "linux" && !cowDisabled.Load() {
|
||||
if err := unix.IoctlFileClone(int(targetFile.Fd()), int(sourceFile.Fd())); err == nil {
|
||||
return nil
|
||||
} else {
|
||||
log.Println(fmt.Errorf("COW attempt for %s failed: %w", targetPath, err))
|
||||
log.Println("Disabling COW until restart")
|
||||
cowDisabled.Store(true)
|
||||
}
|
||||
}
|
||||
|
||||
copySize, err := io.Copy(targetFile, sourceFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("copy source file: %w", err)
|
||||
|
||||
@@ -17,6 +17,11 @@ import (
|
||||
"github.com/derfenix/photocatalog/v2/internal/metadata"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultDirMode = 0o777
|
||||
defaultFileMode = 0o644
|
||||
)
|
||||
|
||||
type MetaExtractor interface {
|
||||
Extract(_ string, data io.Reader) (metadata.Metadata, error)
|
||||
}
|
||||
@@ -31,8 +36,8 @@ func NewOrganizer(mode Mode, source, target string) *Organizer {
|
||||
sourceDir: source,
|
||||
targetDir: target,
|
||||
|
||||
dirMode: 0777,
|
||||
fileMode: 0644,
|
||||
dirMode: defaultDirMode,
|
||||
fileMode: defaultFileMode,
|
||||
|
||||
metaExtractors: map[string]MetaExtractor{
|
||||
"": &metadata.Default{},
|
||||
@@ -268,6 +273,7 @@ func (o *Organizer) BuildTargetPath(sourcePath string, meta metadata.Metadata) (
|
||||
o.targetDir,
|
||||
strconv.Itoa(meta.Created.Year()),
|
||||
strconv.Itoa(int(meta.Created.Month())),
|
||||
strconv.Itoa(meta.Created.Day()),
|
||||
sourcePath,
|
||||
)
|
||||
|
||||
|
||||
17
main.go
17
main.go
@@ -33,25 +33,18 @@ func main() {
|
||||
}
|
||||
|
||||
func loadCfg() application.Config {
|
||||
cfg := application.Config{
|
||||
SourceDir: "",
|
||||
TargetDir: "",
|
||||
Mode: "",
|
||||
Overwrite: false,
|
||||
DirMode: 0,
|
||||
FileMode: 0,
|
||||
Watch: false,
|
||||
}
|
||||
cfg := application.Config{}
|
||||
|
||||
flag.StringVar(&cfg.SourceDir, "source", "", "Source directory")
|
||||
flag.StringVar(&cfg.TargetDir, "target", "", "Target directory")
|
||||
flag.BoolVar(&cfg.Overwrite, "overwrite", false, "Overwrite existing files")
|
||||
flag.BoolVar(&cfg.Watch, "watch", false, "Watch for changes in the source directory")
|
||||
flag.BoolVar(&cfg.Watch, "watch", true, "Watch for changes in the source directory")
|
||||
flag.BoolVar(&cfg.SkipFullSync, "skip-full-sync", false, "Skip full sync at startup")
|
||||
|
||||
var dirMode string
|
||||
var fileMode string
|
||||
flag.StringVar(&dirMode, "dirmode", "0777", "Mode bits for directories can be created while syncing")
|
||||
flag.StringVar(&fileMode, "filemode", "0644", "Mode bits for files created while syncing (not applicable for hardlink mode)")
|
||||
flag.StringVar(&dirMode, "dir-mode", "0777", "Mode bits for directories can be created while syncing")
|
||||
flag.StringVar(&fileMode, "file-mode", "0644", "Mode bits for files created while syncing (not applicable for hardlink mode)")
|
||||
|
||||
var mode string
|
||||
flag.StringVar(&mode, "mode", "hardlink", "Mode")
|
||||
|
||||
Reference in New Issue
Block a user