hjp
/
simba
1
0
Fork 0
Code Issues 2 Pull Requests Releases Wiki Activity
157 Commits 1 Branch 0 Tags 344 KiB
Commit Graph

81 Commits

Author SHA1 Message Date
Peter J. Holzer 45095845fd Limit instances to deduplicate by id 
The instances table is partitioned by id. If we restrinct the search to
recent ids we only have to search a few partitions which should be
faster.
 2023-11-29 06:04:02 +01:00
Peter J. Holzer 118bb0a2e7 Fix pruning 2023-11-29 06:03:36 +01:00
Peter J. Holzer e8c34ce1b5 Fix typo 2022-03-13 23:22:24 +01:00
Peter J. Holzer 7180c24cc3 Add (hardcoded) timeout to reserve_fileset 2021-07-25 12:22:54 +02:00
Peter J. Holzer 88f226757d Add sortorder to filesets 2021-07-25 12:22:27 +02:00
Peter J. Holzer 24932a63ac Log selected previous backup 2021-07-25 12:21:10 +02:00
Peter J. Holzer 289670b313 Treat zero length files specially for better performance 2021-07-25 12:19:18 +02:00
Peter J. Holzer 19b0faad02 Add linkdup_db to statistics 2021-07-25 12:18:09 +02:00
Peter J. Holzer 270c64e758 Log statistics in backup2disk 
Backups are running in child processes so the CA object in the parent is
never updated. And per-fileset statistics are more interesting than the
total anyway.
 2021-01-31 13:47:59 +01:00
Peter J. Holzer 0cbf922eda Log progress by time instead of count 
Instead of logging the current file name every 1000 files we now log it
every 10 seconds. This makes it easier to see where backup is slow.
 2021-01-31 13:39:59 +01:00
Peter J. Holzer 164e6ab553 Log file version id 2021-01-31 13:38:50 +01:00
Peter J. Holzer 23998cefbd Clamp mtime to +/- 2**31. 2021-01-31 13:36:33 +01:00
hjp 34ce8eb0bc Implement timeout 2021-01-21 15:34:30 +00:00
hjp d79a7d083c Fix sessions_on_same_device cache 
This hasn't been working for at least 10 years. We treated the contents
of the cache as a list of ids, but it was really a list of records. Now
we just store the ids, since we don't need the rest of the info.
 2020-07-18 08:52:00 +00:00
hjp f26d6b6640 Run backups for different filesets in parallel 2019-11-09 10:22:34 +00:00
hjp 8aa0c9ca5e Fix typo 2019-09-28 09:15:21 +00:00
hjp a6b9dff48e Don't record fs_id:inode_number and time of backup for each instance by default 
We don't actually use that information.
 2019-09-08 09:05:40 +00:00
hjp 18c9936606 Use O_NOATIME 2018-03-05 19:33:19 +00:00
hjp a83aa49a89 Export to local disk instead of backup medium. 
The export directory will then be backed up normally, which is a simple
copy operation, while exporting all sessions is expensive and should not
be repeated for every new backup medium.
 2017-01-28 14:06:34 +00:00
hjp f8ef39ae0f Add column active to mark filesets which should be backed up. 
This can still be overridden with the filesets option.
 2016-06-08 19:48:33 +00:00
hjp 6514b6cdb6 Create export dir and commit after session. 2016-05-08 19:53:26 +00:00
hjp 810552e10f hardcoded path to gzip. 2014-01-12 11:00:57 +00:00
hjp 6392f06826 At the end of a run, export all sessions which haven't been exported yet 
to the current basedir.
 2014-01-11 18:50:14 +00:00
hjp 2bc5967e10 Automatically create a new partition if the current one doesn't have 
enough free space.
 2013-11-26 21:33:11 +00:00
hjp d360ae4722 Avoid trying to use an undefined value as an array reference. 2013-11-13 22:09:29 +00:00
hjp 317671456d Added Simba/CA/DBI.pm and convert_mysql_to_pgsql to repository. 2013-11-13 00:24:35 +00:00
hjp ff9f0d48f1 Need to commit periodically. flush_insert_instances is the most logical 
place although that may be a bit too often.
 2012-10-09 06:57:27 +00:00
hjp cc770cdf9e Read all checksums from the last session and store them in a hash. 
This should speed up retrieving checksums considerably.
 2012-09-29 10:44:10 +00:00
hjp 069d20a488 replaced Digest::SHA1 with Digest::SHA. 2012-09-20 10:47:19 +00:00
hjp 4f7aa4cc04 it's not (generally) "my" pid 2010-09-11 09:59:24 +00:00
hjp 2596541fd0 rlease lock at the end, don't try to reserve it again. 2010-09-11 07:26:27 +00:00
hjp ea76362546 lock fileset by storing the pid in the table. Somewhat ugly but should 
work.
 2010-09-02 12:05:59 +00:00
hjp 3a2570be8e fixed some typos 2010-09-02 11:41:21 +00:00
hjp 4f4e5540ff lock fileset during backup to avoid concurrent backups of the same 
dataset.
Only search the last few (well, currently 30) sessions with a backup on
the same device for matching files in linkdup.
Started to work on support for tokyocabinet
 2010-09-02 08:24:07 +00:00
hjp 259d97bc99 avoid temporary filenames with more than 255 characters. 2010-05-26 20:09:05 +00:00
hjp 58b8ab7403 finer grained profiling of versions2 handling 2010-05-20 09:24:54 +00:00
hjp 1452afe110 added comment about ENOENT 2010-05-18 08:03:45 +00:00
hjp 28745ebffa documented log levels 2010-04-24 20:23:21 +00:00
hjp f8856de2c9 (late commit) 
Improved search for last session (now only finds sessions which were
completed successfully).
Some speed optimizations (not very successfull, iirc).
 2010-04-09 20:28:20 +00:00
hjp 2bc2878dd3 more timing info 2009-09-25 19:58:20 +00:00
hjp 74da594a66 Only try to hardlink to old versions on the same device. 
This should significantly speed up the first backup to a new device.
Added some timing info to the log.
 2009-09-10 16:14:21 +00:00
hjp b8210f431e Fixed equality checking for setuid files. 
Setuid files are now stored as setuid which means that the backup volume
should be mounted nosuid or even noexec unless you trust all your
clients.
 2009-06-28 20:24:18 +00:00
hjp 0b5aa8283b noted that dieing is suboptimal. 2008-07-15 21:18:16 +00:00
hjp 2edaa67ccc mkdir_p problem 2008-06-20 06:19:18 +00:00
hjp 30512f5471 fixed some db queries. 
Identified (but haven't yet fixed) lack of error handling when reading
from CA.
 2008-05-04 13:42:31 +00:00
hjp 6ce3a6b0c5 somewhat naive conversion to new db layout 2008-04-17 21:34:39 +00:00
hjp 066869e9c2 Try to be more intelligent in linking to old instances of the same file. 
Especially recover from various errors (e.g. link count exvceeded) by
creating a new file.

Note: This code is extremely slow and should not be used. A
reorganisation of the database layout is in progress.
 2008-04-13 10:36:46 +00:00
hjp c42aeb5aee Refactoring: Moved code to store a regular file in a sub of its own, 
but don't change the logic yet. Put info about file related subprocesses 
into $self so that this can be cleaned up after the session ends.
Added a few log messages.
 2008-04-05 11:21:28 +00:00
hjp 446c3a6fdc fixed name of DA config file. 2008-03-23 16:30:48 +00:00
hjp 26afacf9d0 really fix permissions of da 2008-03-21 21:39:32 +00:00