Commit Graph

72 Commits

Author SHA1 Message Date
Peter J. Holzer 0cbf922eda Log progress by time instead of count
Instead of logging the current file name every 1000 files we now log it
every 10 seconds. This makes it easier to see where backup is slow.
2021-01-31 13:39:59 +01:00
Peter J. Holzer 164e6ab553 Log file version id 2021-01-31 13:38:50 +01:00
Peter J. Holzer 23998cefbd Clamp mtime to +/- 2**31. 2021-01-31 13:36:33 +01:00
hjp 34ce8eb0bc Implement timeout 2021-01-21 15:34:30 +00:00
hjp d79a7d083c Fix sessions_on_same_device cache
This hasn't been working for at least 10 years. We treated the contents
of the cache as a list of ids, but it was really a list of records. Now
we just store the ids, since we don't need the rest of the info.
2020-07-18 08:52:00 +00:00
hjp f26d6b6640 Run backups for different filesets in parallel 2019-11-09 10:22:34 +00:00
hjp 8aa0c9ca5e Fix typo 2019-09-28 09:15:21 +00:00
hjp a6b9dff48e Don't record fs_id:inode_number and time of backup for each instance by default
We don't actually use that information.
2019-09-08 09:05:40 +00:00
hjp 18c9936606 Use O_NOATIME 2018-03-05 19:33:19 +00:00
hjp a83aa49a89 Export to local disk instead of backup medium.
The export directory will then be backed up normally, which is a simple
copy operation, while exporting all sessions is expensive and should not
be repeated for every new backup medium.
2017-01-28 14:06:34 +00:00
hjp f8ef39ae0f Add column active to mark filesets which should be backed up.
This can still be overridden with the filesets option.
2016-06-08 19:48:33 +00:00
hjp 6514b6cdb6 Create export dir and commit after session. 2016-05-08 19:53:26 +00:00
hjp 810552e10f hardcoded path to gzip. 2014-01-12 11:00:57 +00:00
hjp 6392f06826 At the end of a run, export all sessions which haven't been exported yet
to the current basedir.
2014-01-11 18:50:14 +00:00
hjp 2bc5967e10 Automatically create a new partition if the current one doesn't have
enough free space.
2013-11-26 21:33:11 +00:00
hjp d360ae4722 Avoid trying to use an undefined value as an array reference. 2013-11-13 22:09:29 +00:00
hjp 317671456d Added Simba/CA/DBI.pm and convert_mysql_to_pgsql to repository. 2013-11-13 00:24:35 +00:00
hjp ff9f0d48f1 Need to commit periodically. flush_insert_instances is the most logical
place although that may be a bit too often.
2012-10-09 06:57:27 +00:00
hjp cc770cdf9e Read all checksums from the last session and store them in a hash.
This should speed up retrieving checksums considerably.
2012-09-29 10:44:10 +00:00
hjp 069d20a488 replaced Digest::SHA1 with Digest::SHA. 2012-09-20 10:47:19 +00:00
hjp 4f7aa4cc04 it's not (generally) "my" pid 2010-09-11 09:59:24 +00:00
hjp 2596541fd0 rlease lock at the end, don't try to reserve it again. 2010-09-11 07:26:27 +00:00
hjp ea76362546 lock fileset by storing the pid in the table. Somewhat ugly but should
work.
2010-09-02 12:05:59 +00:00
hjp 3a2570be8e fixed some typos 2010-09-02 11:41:21 +00:00
hjp 4f4e5540ff lock fileset during backup to avoid concurrent backups of the same
dataset.
Only search the last few (well, currently 30) sessions with a backup on
the same device for matching files in linkdup.
Started to work on support for tokyocabinet
2010-09-02 08:24:07 +00:00
hjp 259d97bc99 avoid temporary filenames with more than 255 characters. 2010-05-26 20:09:05 +00:00
hjp 58b8ab7403 finer grained profiling of versions2 handling 2010-05-20 09:24:54 +00:00
hjp 1452afe110 added comment about ENOENT 2010-05-18 08:03:45 +00:00
hjp 28745ebffa documented log levels 2010-04-24 20:23:21 +00:00
hjp f8856de2c9 (late commit)
Improved search for last session (now only finds sessions which were
completed successfully).
Some speed optimizations (not very successfull, iirc).
2010-04-09 20:28:20 +00:00
hjp 2bc2878dd3 more timing info 2009-09-25 19:58:20 +00:00
hjp 74da594a66 Only try to hardlink to old versions on the same device.
This should significantly speed up the first backup to a new device.
Added some timing info to the log.
2009-09-10 16:14:21 +00:00
hjp b8210f431e Fixed equality checking for setuid files.
Setuid files are now stored as setuid which means that the backup volume
should be mounted nosuid or even noexec unless you trust all your
clients.
2009-06-28 20:24:18 +00:00
hjp 0b5aa8283b noted that dieing is suboptimal. 2008-07-15 21:18:16 +00:00
hjp 2edaa67ccc mkdir_p problem 2008-06-20 06:19:18 +00:00
hjp 30512f5471 fixed some db queries.
Identified (but haven't yet fixed) lack of error handling when reading
from CA.
2008-05-04 13:42:31 +00:00
hjp 6ce3a6b0c5 somewhat naive conversion to new db layout 2008-04-17 21:34:39 +00:00
hjp 066869e9c2 Try to be more intelligent in linking to old instances of the same file.
Especially recover from various errors (e.g. link count exvceeded) by
creating a new file.

Note: This code is extremely slow and should not be used. A
reorganisation of the database layout is in progress.
2008-04-13 10:36:46 +00:00
hjp c42aeb5aee Refactoring: Moved code to store a regular file in a sub of its own,
but don't change the logic yet. Put info about file related subprocesses 
into $self so that this can be cleaned up after the session ends.
Added a few log messages.
2008-04-05 11:21:28 +00:00
hjp 446c3a6fdc fixed name of DA config file. 2008-03-23 16:30:48 +00:00
hjp 26afacf9d0 really fix permissions of da 2008-03-21 21:39:32 +00:00
hjp 3fa026c805 Added option filesets to backup only some filesets (instead of all of
them).
2008-03-21 11:12:42 +00:00
hjp b32e22fd99 fix permissions of da. 2008-03-20 21:13:04 +00:00
hjp 579b0a33b3 Find duplicate files even if file name is different. 2007-11-17 21:03:10 +00:00
hjp 474ef9e7ef Moved prune list of disk agent to a config file. 2007-11-17 21:01:51 +00:00
hjp 36d9cdef22 copy checksum from previous session if file hasn't changed. 2007-11-15 20:17:55 +00:00
hjp a8634adc3b Die if disk full. 2007-06-22 20:41:56 +00:00
hjp 719cc67c25 record sessions. 2007-06-18 20:11:27 +00:00
hjp f087e0f535 Don't backup /backup. 2007-06-18 19:43:53 +00:00
hjp 0475702456 Get targets from database.
Store metadata of backed up files in database.
Use -i option of ssh.
2007-06-18 19:42:08 +00:00