mirror of
https://github.com/derfenix/photocatalog.git
synced 2026-03-11 21:35:34 +03:00
Compare commits
9 Commits
v2.0.0-alp
...
3ae91523d8
| Author | SHA1 | Date | |
|---|---|---|---|
| 3ae91523d8 | |||
| 535c790c39 | |||
| 73aa5c4ac2 | |||
| 077f920986 | |||
| 77443e2369 | |||
| 9acb554dd4 | |||
| e3f8f7b8c8 | |||
| 62ca9f2378 | |||
| f73d612666 |
28
.github/workflows/go.yml
vendored
Normal file
28
.github/workflows/go.yml
vendored
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
# This workflow will build a golang project
|
||||||
|
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go
|
||||||
|
|
||||||
|
name: Go
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [ "master" ]
|
||||||
|
pull_request:
|
||||||
|
branches: [ "master" ]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
|
||||||
|
build:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Set up Go
|
||||||
|
uses: actions/setup-go@v4
|
||||||
|
with:
|
||||||
|
go-version: '1.22'
|
||||||
|
|
||||||
|
- name: Build
|
||||||
|
run: go build -v ./...
|
||||||
|
|
||||||
|
- name: Test
|
||||||
|
run: go test -v ./...
|
||||||
75
README.md
75
README.md
@@ -1,25 +1,23 @@
|
|||||||
# Simple photo cataloguer
|
[](https://github.com/derfenix/photocatalog/actions/workflows/go.yml)
|
||||||
|
|
||||||
|
# Effortless Photo Organizer
|
||||||
|
|
||||||
Just copy/hardlink photos (or video, or any other files) from one place to
|
Just copy/hardlink photos (or video, or any other files) from one place to
|
||||||
another, separating them in sub-directories like `$ROOT/year/month/day/`.
|
another, separating them in sub-directories like `$ROOT/year/month/day/`.
|
||||||
|
|
||||||
### TL;DR
|
### TL;DR
|
||||||
|
|
||||||
I have a smartphone, I have a Syncthing ~~uugh... SmartThing~~ and all photos
|
I use a smartphone along with Syncthing to seamlessly sync all my photos to my PC without any manual effort. However, there's a catch: I can't keep all my photos in the synced folder indefinitely. If I clear my phone's memory, the photos on my PC get deleted as well. To avoid this, I need to remember to copy the files to another location before cleaning up my phone.
|
||||||
from smartphone nicely synced to my PC without my attention. But I can't just
|
|
||||||
keep all photos in synced folder: if I'll clean my phone memory - all photos
|
|
||||||
from pc will be cleaned too. I need to not forget copy files in another
|
|
||||||
place before cleaning phone's memory. Also, I can't just drop all photos in
|
|
||||||
one dir - I will not find anything there later, and a folder with thousands
|
|
||||||
photos looks like a bad idea from either side.
|
|
||||||
So I create this tool in one evening. All it does - copy (or create hardlinks for)
|
|
||||||
files from one place to another, creating basic date-aware directories
|
|
||||||
structure for that files.
|
|
||||||
|
|
||||||
|
Simply dumping all my photos into one folder isn't a solution either — finding anything later would be a nightmare, and a folder with thousands of unsorted photos is far from ideal.
|
||||||
|
|
||||||
|
To address these issues, I created this tool in just one evening. Its primary purpose is to copy (or create hardlinks for) files from one location to another, while organizing them into a simple, date-based directory structure.
|
||||||
|
|
||||||
|
This tool was built for personal use and has been serving me well for quite some time without any problems. However, if you encounter any issues, feel free to report them — I’d be happy to help.
|
||||||
|
|
||||||
## Installing
|
## Installing
|
||||||
```bash
|
```bash
|
||||||
go install github.com/derfenix/photocatalog@latest
|
go install github.com/derfenix/photocatalog/v2@latest
|
||||||
```
|
```
|
||||||
Optionally you could copy created binary from the GO's bin path to
|
Optionally you could copy created binary from the GO's bin path to
|
||||||
system or user $PATH, e.g. /usr/local/bin/.
|
system or user $PATH, e.g. /usr/local/bin/.
|
||||||
@@ -27,44 +25,63 @@ system or user $PATH, e.g. /usr/local/bin/.
|
|||||||
sudo cp ${GOPATH}/bin/photocatalog /usr/local/bin/photocatalog
|
sudo cp ${GOPATH}/bin/photocatalog /usr/local/bin/photocatalog
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Migrating from v0.*
|
||||||
|
|
||||||
|
TODO
|
||||||
|
|
||||||
|
## Organization modes
|
||||||
|
|
||||||
|
Next organization modes supported:
|
||||||
|
|
||||||
|
- **copy** — copy files to target root. Make COW (using syscall) if FS supports it.
|
||||||
|
- **hardlink** — create hardlink to the source file instead of copying.
|
||||||
|
The best choice if source and target are in same partition for compatibility
|
||||||
|
and resource usage, but we can't chmod target files, because of original file mode will
|
||||||
|
be changed too.
|
||||||
|
- **move** — moves original files to new place.
|
||||||
|
- **symlink** — create a symlink at the target for the source files.
|
||||||
|
|
||||||
## Supported formats
|
## Supported formats
|
||||||
At this moment supported jpeg files with filled exif data or any other
|
At this moment supported jpeg and tiff files with filled exif data and any other
|
||||||
files but with names matching pattern `yyymmdd_HHMMSS.ext`. Such
|
files but with names matching pattern `yyymmdd_HHMMSS.ext` with optional suffixes after a timestamp.
|
||||||
names format applied by android's camera software (I guess all cams
|
Such names format applied by the Android's camera software (I guess all cams
|
||||||
use this format, fix me if I'm wrong).
|
use this format, fix me if I'm wrong).
|
||||||
|
|
||||||
There is no support for changing names format without modifying source code
|
Jpeg/Tiff files without modification date if exif will be fallen back to the name parsing.
|
||||||
at this time.
|
|
||||||
|
No able to change names format without modifying source code for now. Just because
|
||||||
|
I have reasons to believe that this format is the most popular for the application use cases.
|
||||||
|
But let me know if you need different timestamp formats support.
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
### One-shot
|
### One-shot
|
||||||
#### Copy files (make a COW if fs supports it)
|
#### Copy files
|
||||||
```bash
|
```bash
|
||||||
photocalog -mode copy -target ./photos/ ./sync/photos/*
|
photocalog -mode copy -target ./photos/ -source ./sync/photos/
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Create hardlinks (only withing one disk partition)
|
#### Create hardlinks
|
||||||
```bash
|
```bash
|
||||||
photocalog -mode hardlink -target ./photos/ ./sync/photos/*
|
photocalog -mode hardlink -target ./photos/ -source ./sync/photos/
|
||||||
```
|
```
|
||||||
or
|
or
|
||||||
```bash
|
```bash
|
||||||
photocalog -target ./photos/ ./sync/photos/*
|
photocalog -target ./photos/ -source ./sync/photos/*
|
||||||
```
|
```
|
||||||
|
|
||||||
### Monitor
|
### Watch mode
|
||||||
#### Copy files (make a COW if fs supports it)
|
#### Copy files
|
||||||
```bash
|
```bash
|
||||||
photocalog -mode copy -target ./photos -monitor ./sync/photos/*
|
photocalog -mode copy -target ./photos -watch -source ./sync/photos/
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Create hardlinks (only withing one disk partition)
|
#### Create hardlinks
|
||||||
```bash
|
```bash
|
||||||
photocalog -mode hardlink -target ./photos/ -monitor ./sync/photos/
|
photocalog -mode hardlink -target ./photos/ -watch -source ./sync/photos/
|
||||||
```
|
```
|
||||||
or
|
or
|
||||||
```bash
|
```bash
|
||||||
photocalog -target ./photos/ -monitor ./sync/photos/
|
photocalog -target ./photos/ -watch -source ./sync/photos/
|
||||||
```
|
```
|
||||||
|
|
||||||
## Install and run monitor service
|
## Install and run monitor service
|
||||||
@@ -89,6 +106,6 @@ under corresponding sub-dir.
|
|||||||
|
|
||||||
### Why this tool was created if there is awesome XXX tool?
|
### Why this tool was created if there is awesome XXX tool?
|
||||||
I had two good reasons:
|
I had two good reasons:
|
||||||
1. I wanted
|
1. I wish
|
||||||
2. I can
|
2. I can
|
||||||
|
|
||||||
|
|||||||
@@ -50,8 +50,10 @@ func (a *Application) Start(ctx context.Context, wg *sync.WaitGroup) error {
|
|||||||
org = org.WithOverwrite()
|
org = org.WithOverwrite()
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := org.FullSync(ctx); err != nil {
|
if !a.config.SkipFullSync {
|
||||||
return fmt.Errorf("full sync: %w", err)
|
if err := org.FullSync(ctx); err != nil {
|
||||||
|
return fmt.Errorf("full sync: %w", err)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if a.config.Watch {
|
if a.config.Watch {
|
||||||
|
|||||||
@@ -15,13 +15,14 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
type Config struct {
|
type Config struct {
|
||||||
SourceDir string
|
SourceDir string
|
||||||
TargetDir string
|
TargetDir string
|
||||||
Mode Mode
|
Mode Mode
|
||||||
Overwrite bool
|
Overwrite bool
|
||||||
DirMode uint64
|
DirMode uint64
|
||||||
FileMode uint64
|
FileMode uint64
|
||||||
Watch bool
|
Watch bool
|
||||||
|
SkipFullSync bool
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Config) Validate() error {
|
func (c *Config) Validate() error {
|
||||||
@@ -33,9 +34,13 @@ func (c *Config) Validate() error {
|
|||||||
return fmt.Errorf("target dir is required")
|
return fmt.Errorf("target dir is required")
|
||||||
}
|
}
|
||||||
|
|
||||||
if !slices.Contains([]Mode{ModeHardlink, ModeSymlink, ModeMove, ModeSymlink}, c.Mode) {
|
if !slices.Contains([]Mode{ModeHardlink, ModeSymlink, ModeMove, ModeCopy}, c.Mode) {
|
||||||
return fmt.Errorf("invalid mode %s", c.Mode)
|
return fmt.Errorf("invalid mode %s", c.Mode)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if c.SkipFullSync && !c.Watch {
|
||||||
|
return fmt.Errorf("skip full sync and watch disabled — nothing to do")
|
||||||
|
}
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,12 +5,18 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log"
|
"log"
|
||||||
"os"
|
"os"
|
||||||
|
"runtime"
|
||||||
|
"sync/atomic"
|
||||||
|
|
||||||
|
"golang.org/x/sys/unix"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
var cowDisabled = atomic.Bool{}
|
||||||
|
|
||||||
type Copy struct{}
|
type Copy struct{}
|
||||||
|
|
||||||
func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
|
func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
|
||||||
targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_RDWR|os.O_CREATE, mode)
|
targetFile, err := os.OpenFile(targetPath, os.O_TRUNC|os.O_WRONLY|os.O_CREATE, mode)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("open target file: %w", err)
|
return fmt.Errorf("open target file: %w", err)
|
||||||
}
|
}
|
||||||
@@ -28,6 +34,17 @@ func (c Copy) PlaceIt(sourcePath, targetPath string, mode os.FileMode) error {
|
|||||||
_ = sourceFile.Close()
|
_ = sourceFile.Close()
|
||||||
}()
|
}()
|
||||||
|
|
||||||
|
// Try to do a COW.
|
||||||
|
if runtime.GOOS == "linux" && !cowDisabled.Load() {
|
||||||
|
if err := unix.IoctlFileClone(int(targetFile.Fd()), int(sourceFile.Fd())); err == nil {
|
||||||
|
return nil
|
||||||
|
} else {
|
||||||
|
log.Println(fmt.Errorf("COW attempt for %s failed: %w", targetPath, err))
|
||||||
|
log.Println("Disabling COW until restart")
|
||||||
|
cowDisabled.Store(true)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
copySize, err := io.Copy(targetFile, sourceFile)
|
copySize, err := io.Copy(targetFile, sourceFile)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("copy source file: %w", err)
|
return fmt.Errorf("copy source file: %w", err)
|
||||||
|
|||||||
@@ -17,6 +17,11 @@ import (
|
|||||||
"github.com/derfenix/photocatalog/v2/internal/metadata"
|
"github.com/derfenix/photocatalog/v2/internal/metadata"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
defaultDirMode = 0o777
|
||||||
|
defaultFileMode = 0o644
|
||||||
|
)
|
||||||
|
|
||||||
type MetaExtractor interface {
|
type MetaExtractor interface {
|
||||||
Extract(_ string, data io.Reader) (metadata.Metadata, error)
|
Extract(_ string, data io.Reader) (metadata.Metadata, error)
|
||||||
}
|
}
|
||||||
@@ -31,8 +36,8 @@ func NewOrganizer(mode Mode, source, target string) *Organizer {
|
|||||||
sourceDir: source,
|
sourceDir: source,
|
||||||
targetDir: target,
|
targetDir: target,
|
||||||
|
|
||||||
dirMode: 0777,
|
dirMode: defaultDirMode,
|
||||||
fileMode: 0644,
|
fileMode: defaultFileMode,
|
||||||
|
|
||||||
metaExtractors: map[string]MetaExtractor{
|
metaExtractors: map[string]MetaExtractor{
|
||||||
"": &metadata.Default{},
|
"": &metadata.Default{},
|
||||||
@@ -268,6 +273,7 @@ func (o *Organizer) BuildTargetPath(sourcePath string, meta metadata.Metadata) (
|
|||||||
o.targetDir,
|
o.targetDir,
|
||||||
strconv.Itoa(meta.Created.Year()),
|
strconv.Itoa(meta.Created.Year()),
|
||||||
strconv.Itoa(int(meta.Created.Month())),
|
strconv.Itoa(int(meta.Created.Month())),
|
||||||
|
strconv.Itoa(meta.Created.Day()),
|
||||||
sourcePath,
|
sourcePath,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
17
main.go
17
main.go
@@ -33,25 +33,18 @@ func main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func loadCfg() application.Config {
|
func loadCfg() application.Config {
|
||||||
cfg := application.Config{
|
cfg := application.Config{}
|
||||||
SourceDir: "",
|
|
||||||
TargetDir: "",
|
|
||||||
Mode: "",
|
|
||||||
Overwrite: false,
|
|
||||||
DirMode: 0,
|
|
||||||
FileMode: 0,
|
|
||||||
Watch: false,
|
|
||||||
}
|
|
||||||
|
|
||||||
flag.StringVar(&cfg.SourceDir, "source", "", "Source directory")
|
flag.StringVar(&cfg.SourceDir, "source", "", "Source directory")
|
||||||
flag.StringVar(&cfg.TargetDir, "target", "", "Target directory")
|
flag.StringVar(&cfg.TargetDir, "target", "", "Target directory")
|
||||||
flag.BoolVar(&cfg.Overwrite, "overwrite", false, "Overwrite existing files")
|
flag.BoolVar(&cfg.Overwrite, "overwrite", false, "Overwrite existing files")
|
||||||
flag.BoolVar(&cfg.Watch, "watch", false, "Watch for changes in the source directory")
|
flag.BoolVar(&cfg.Watch, "watch", true, "Watch for changes in the source directory")
|
||||||
|
flag.BoolVar(&cfg.SkipFullSync, "skip-full-sync", false, "Skip full sync at startup")
|
||||||
|
|
||||||
var dirMode string
|
var dirMode string
|
||||||
var fileMode string
|
var fileMode string
|
||||||
flag.StringVar(&dirMode, "dirmode", "0777", "Mode bits for directories can be created while syncing")
|
flag.StringVar(&dirMode, "dir-mode", "0777", "Mode bits for directories can be created while syncing")
|
||||||
flag.StringVar(&fileMode, "filemode", "0644", "Mode bits for files created while syncing (not applicable for hardlink mode)")
|
flag.StringVar(&fileMode, "file-mode", "0644", "Mode bits for files created while syncing (not applicable for hardlink mode)")
|
||||||
|
|
||||||
var mode string
|
var mode string
|
||||||
flag.StringVar(&mode, "mode", "hardlink", "Mode")
|
flag.StringVar(&mode, "mode", "hardlink", "Mode")
|
||||||
|
|||||||
Reference in New Issue
Block a user