Add command to verify an archive

Post here if you want to request a features that has not been implemented yet
Post Reply
mbiebl
Posts: 12
Joined: Sat Nov 28, 2009 8:02 pm

Add command to verify an archive

Post by mbiebl » Sat Nov 28, 2009 8:06 pm

Hi,

I think it would be helpful if fsarchiver provided a command which let's you verify if an archive is correct, i.e. not corrupt.

admin
Site Admin
Posts: 550
Joined: Sat Feb 21, 2004 12:12 pm

Re: Add command to verify an archive

Post by admin » Sun Nov 29, 2009 11:25 am

Yes that's an important feature. It can be implemented by doing a restfs that does not write anything on the disk. I have that in mind for the future.

Coastie
Posts: 18
Joined: Wed Dec 16, 2009 8:18 pm

Re: Add command to verify an archive

Post by Coastie » Wed Dec 16, 2009 8:20 pm

This is a feature I am looking for too. As a replacement for partimage, I'd like to use fsarchiver, but would like to know that the backup I am creating and relying on is valid.

admin
Site Admin
Posts: 550
Joined: Sat Feb 21, 2004 12:12 pm

Re: Add command to verify an archive

Post by admin » Wed Dec 16, 2009 10:42 pm

I have that in mind.

Coastie
Posts: 18
Joined: Wed Dec 16, 2009 8:18 pm

Re: Add command to verify an archive

Post by Coastie » Sun Feb 28, 2010 7:23 pm

Has any progress been made on this feature?

admin
Site Admin
Posts: 550
Joined: Sat Feb 21, 2004 12:12 pm

Re: Add command to verify an archive

Post by admin » Sun Feb 28, 2010 8:10 pm

I am currently working on savept / restpt to save/restore the partition table in 0.7.0, am just doing one thing at a time.

Coastie
Posts: 18
Joined: Wed Dec 16, 2009 8:18 pm

Re: Add command to verify an archive

Post by Coastie » Sat Jun 19, 2010 12:06 am

Is there any new information on a way to verify/test an archive? Is it included in the 0.7.0 betas?

admin
Site Admin
Posts: 550
Joined: Sat Feb 21, 2004 12:12 pm

Re: Add command to verify an archive

Post by admin » Mon Jun 21, 2010 12:24 pm

Version 0.7.0 only has the savept/restpt stuff now. I will try to finish that and then I will focus on other features, but I first want to finish this. I have been very busy on stabilizing SystemRescueCd-1.5.x in the past few months, not sure when I will have time to make some progress on fsarchiver-0.7.x. The most important is to have a stable fsarchiver-0.6.x and SystemRescueCd-1.5.x anyway.

misterc
Posts: 1
Joined: Tue Dec 20, 2011 8:24 pm

Re: Add command to verify an archive

Post by misterc » Tue Dec 20, 2011 8:45 pm

creating a file as block device allows to restore a filesystem to a file, provided one has enough disk space.

Code: Select all

/ # dd if=/dev/zero of=/mnt/path/test_fsa count=46000k               # size = 24117248000 bytes
/ # losetup /dev/loop0 /mnt/path/test_fsa                            # makes the file appear like a block device
/ # fdisk /dev/loop0                                                     # create partition as needed
/ # fsarchiver restfs -j{# of cores} /mnt/path/to/root_z7_20111220.fsa id=0,dest=/dev/loop0
/ # rm /mnt/path/test_fsa                                      # considering the size...
it is a workaround to not have a check functionality provided, but for someone who only cares about backing up one's system, the performance gain thanks to the -j option is more then lost by having to actually write the data back somewhere instead of being able to check it :(

using /dev/null is not an option as it isn't a block device

maybe in a version 0.70?

oliver
Posts: 3
Joined: Wed Jul 03, 2013 9:55 pm

Re: Add command to verify an archive

Post by oliver » Wed Jul 03, 2013 10:05 pm

Hi, this is a patch for a verify operation on fsarchiver-0.6.17. It seems to work, but not yet extensively tested.

BR, Oliver

Code: Select all

From 79c3a0d0a2bdc2bdc8ea7edc68140fb1a28006f1 Mon Sep 17 00:00:00 2001
From: Oliver Winker <[email protected]>
Date: Wed, 3 Jul 2013 23:45:18 +0200
Subject: [PATCH] Add verify oper for ARCHTYPE_FILESYSTEMS and
 ARCHTYPE_DIRECTORIES

---
 src/fsarchiver.c   |    8 ++
 src/fsarchiver.h   |    2 +-
 src/oper_restore.c |  225 ++++++++++++++++++++++++++++++++++++++++++----------
 3 files changed, 194 insertions(+), 41 deletions(-)

diff --git a/src/fsarchiver.c b/src/fsarchiver.c
index 47084ea..8d92427 100644
--- a/src/fsarchiver.c
+++ b/src/fsarchiver.c
@@ -65,6 +65,7 @@ void usage(char *progname, bool examples)
     msgprintf(MSG_FORCE, " * restdir: restore data from an archive which is not based on a filesystem\n");
     msgprintf(MSG_FORCE, " * archinfo: show information about an existing archive file and its contents\n");
     msgprintf(MSG_FORCE, " * probe [detailed]: show list of filesystems detected on the disks\n");
+    msgprintf(MSG_FORCE, " * verify: verify an archive\n");
     msgprintf(MSG_FORCE, "<options>\n");
     msgprintf(MSG_FORCE, " -o: overwrite the archive if it already exists instead of failing\n");
     msgprintf(MSG_FORCE, " -v: verbose mode (can be used several times to increase the level of details)\n");
@@ -304,6 +305,11 @@ int process_cmdline(int argc, char **argv)
         runasroot=true;
         argcok=(argc<=1);
     }
+    else if (strcmp(command, "verify")==0)
+    {   cmd=OPER_VERIFY;
+        runasroot=false;
+        argcok=(argc<=1);
+    }
     else // command not found
     {   errprintf("[%s] is not a valid command.\n", command);
         usage(progname, false);
@@ -361,6 +367,7 @@ int process_cmdline(int argc, char **argv)
         case OPER_SAVEDIR:
         case OPER_RESTDIR:
         case OPER_ARCHINFO:
+        case OPER_VERIFY:
             archive=*argv++, argc--;
             break;
         case OPER_PROBE:
@@ -399,6 +406,7 @@ int process_cmdline(int argc, char **argv)
         case OPER_RESTFS:
         case OPER_RESTDIR:
         case OPER_ARCHINFO:
+        case OPER_VERIFY:
             ret=oper_restore(archive, fscount, partition, cmd);
             break;
         case OPER_PROBE:
diff --git a/src/fsarchiver.h b/src/fsarchiver.h
index dfd698d..6672c21 100644
--- a/src/fsarchiver.h
+++ b/src/fsarchiver.h
@@ -30,7 +30,7 @@
 #endif
 
 // -------------------------------- fsarchiver commands ---------------------------------------------
-enum {OPER_NULL=0, OPER_SAVEFS, OPER_RESTFS, OPER_SAVEDIR, OPER_RESTDIR, OPER_ARCHINFO, OPER_PROBE};
+enum {OPER_NULL=0, OPER_SAVEFS, OPER_RESTFS, OPER_SAVEDIR, OPER_RESTDIR, OPER_ARCHINFO, OPER_PROBE, OPER_VERIFY};
 
 // ----------------------------------- dico sections ------------------------------------------------
 enum {DICO_OBJ_SECTION_STDATTR=0, DICO_OBJ_SECTION_XATTR=1, DICO_OBJ_SECTION_WINATTR=2};
diff --git a/src/oper_restore.c b/src/oper_restore.c
index 487885b..a5e8896 100644
--- a/src/oper_restore.c
+++ b/src/oper_restore.c
@@ -62,6 +62,8 @@ typedef struct s_extractar
     u64         cost_current;
 } cextractar;
 
+static int is_oper_verify = false;
+
 // returns true if this file of a parent directory has been excluded
 int is_filedir_excluded(char *relpath)
 {
@@ -321,7 +323,10 @@ int extractar_restore_attr_std(cextractar *exar, u32 objtype, char *fullpath, ch
 int extractar_restore_attr_everything(cextractar *exar, int objtype, char *fullpath, char *relpath, cdico *dicoattr)
 {
     int res=0;
-    
+
+    if (is_oper_verify)
+	goto extractar_restore_attr_everything_done;
+
     // ---- restore standard attributes
     res+=extractar_restore_attr_std(exar, objtype, fullpath, relpath, dicoattr);
     
@@ -330,7 +335,8 @@ int extractar_restore_attr_everything(cextractar *exar, int objtype, char *fullp
     
     // ---- restore windows attributes
     res+=extractar_restore_attr_windows(exar, objtype, fullpath, relpath, dicoattr);
-    
+
+extractar_restore_attr_everything_done:
     return (res==0)?(0):(-1);
 }
 
@@ -344,7 +350,10 @@ int extractar_restore_obj_symlink(cextractar *exar, char *fullpath, char *relpat
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
-    
+
+    if (is_oper_verify)
+	goto extractar_restore_obj_symlink_done;
+
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
         goto extractar_restore_obj_symlink_err;
@@ -411,7 +420,8 @@ int extractar_restore_obj_symlink(cextractar *exar, char *fullpath, char *relpat
     {   sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_symlink_err;
     }
-    
+
+extractar_restore_obj_symlink_done:
     dico_destroy(d);
     exar->stats.cnt_symlink++;
     return 0; // success
@@ -432,7 +442,10 @@ int extractar_restore_obj_hardlink(cextractar *exar, char *fullpath, char *relpa
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
-    
+
+    if (is_oper_verify)
+	goto extractar_restore_obj_hardlink_done;
+
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
         goto extractar_restore_obj_hardlink_err;
@@ -469,7 +482,8 @@ int extractar_restore_obj_hardlink(cextractar *exar, char *fullpath, char *relpa
     {   sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_hardlink_err;
     }
-    
+
+extractar_restore_obj_hardlink_done:
     dico_destroy(d);
     exar->stats.cnt_hardlink++;
     return 0; // success
@@ -489,6 +503,9 @@ int extractar_restore_obj_devfile(cextractar *exar, char *fullpath, char *relpat
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
+
+    if (is_oper_verify)
+	goto extractar_restore_obj_devfile_done;
     
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -522,7 +539,8 @@ int extractar_restore_obj_devfile(cextractar *exar, char *fullpath, char *relpat
     {   sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_devfile_err;
     }
-    
+
+extractar_restore_obj_devfile_done:
     dico_destroy(d);
     exar->stats.cnt_special++;
     return 0; // success
@@ -540,6 +558,9 @@ int extractar_restore_obj_directory(cextractar *exar, char *fullpath, char *relp
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
+
+    if (is_oper_verify)
+	goto extractar_restore_obj_directory_done;
     
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -567,7 +588,8 @@ int extractar_restore_obj_directory(cextractar *exar, char *fullpath, char *relp
     {   sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_directory_err;
     }
-    
+
+extractar_restore_obj_directory_done:
     dico_destroy(d);
     exar->stats.cnt_dir++;
     return 0; // success
@@ -600,13 +622,14 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
     s64 lres;
     int res;
     int i;
+    int simul=false;
     
     // init
     errors=0;
     memset(&blkinfo, 0, sizeof(blkinfo));
     regmulti_init(&regmulti, FSA_MAX_BLKSIZE);
     datafile=datafile_alloc();
-    
+
     // ---- dequeue header for each small file which is part of that group
     if (dico_get_u32(dicofirstfile, 0, DISKITEMKEY_MULTIFILESCOUNT, &filescount)!=0)
     {   errprintf("cannot read DISKITEMKEY_MULTIFILESCOUNT from header in archive\n");
@@ -674,15 +697,19 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
         exar->cost_current+=datsize; // filesize
         
         // check the list of excluded files/dirs
-        if (is_filedir_excluded(relpath)!=true)
+        if (is_filedir_excluded(relpath)!=true  || is_oper_verify)
         {
-            // create parent directory if necessary
-            extract_dirpath(fullpath, parentdir, sizeof(parentdir));
-            mkdir_recursive(parentdir);
-            
-            // backup parent dir atime/mtime
-            get_parent_dir_time_attrib(fullpath, parentdir, sizeof(parentdir), tv);
-            
+	    if (is_oper_verify) {
+		simul = true;
+	    } else {
+		// create parent directory if necessary
+		extract_dirpath(fullpath, parentdir, sizeof(parentdir));
+		mkdir_recursive(parentdir);
+
+		// backup parent dir atime/mtime
+		get_parent_dir_time_attrib(fullpath, parentdir, sizeof(parentdir), tv);
+	    }
+
             extractar_listing_print_file(exar, tmpobjtype, relpath);
             
             if (dico_get_data(filehead, DICO_OBJ_SECTION_STDATTR, DISKITEMKEY_MD5SUM, md5sumorig, 16, NULL))
@@ -691,7 +718,7 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
                 goto extractar_restore_obj_regfile_multi_err;
             }
             
-            if (datafile_open_write(datafile, fullpath, false, false)<0)
+            if (datafile_open_write(datafile, fullpath, simul, false)<0)
                 goto extractar_restore_obj_regfile_multi_err;
             
             res=datafile_write(datafile, databuf, datsize);
@@ -709,17 +736,21 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
                 res=truncate(fullpath, 0); // don't leave corrupt data in the file
                 goto extractar_restore_obj_regfile_multi_err;
             }
-                      
-            if (extractar_restore_attr_everything(exar, objtype, fullpath, relpath, filehead)!=0)
-            {   msgprintf(MSG_STACK, "cannot restore file attributes for file [%s]\n", relpath);
-                goto extractar_restore_obj_regfile_multi_err;
-            }
-            
-            // restore parent dir mtime/atime
-            if (utimes(parentdir, tv)!=0)
-            {   sysprintf("utimes(%s) failed\n", parentdir);
-                goto extractar_restore_obj_regfile_multi_err;
-            }
+
+	    if (!is_oper_verify) {
+
+		if (extractar_restore_attr_everything(exar, objtype, fullpath, relpath, filehead)!=0)
+		{   msgprintf(MSG_STACK, "cannot restore file attributes for file [%s]\n", relpath);
+		    goto extractar_restore_obj_regfile_multi_err;
+		}
+
+		// restore parent dir mtime/atime
+		if (utimes(parentdir, tv)!=0)
+		{   sysprintf("utimes(%s) failed\n", parentdir);
+		    goto extractar_restore_obj_regfile_multi_err;
+		}
+
+	    }
             exar->stats.cnt_regfile++;
         }
         
@@ -755,12 +786,13 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
     u64 filepos=0;
     u64 flags=0;
     s64 lres;
-    
+    int simul=false;
+
     // init
     memset(&blkinfo, 0, sizeof(blkinfo));
     memset(magic, 0, sizeof(magic));
     datafile=datafile_alloc();
-    
+
     if (dico_get_u64(d, DICO_OBJ_SECTION_STDATTR, DISKITEMKEY_SIZE, &filesize)!=0)
     {   errprintf("Cannot read filesize DISKITEMKEY_SIZE from archive for file=[%s]\n", relpath);
         minorerr=true;
@@ -773,9 +805,13 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
     exar->cost_current+=filesize;
     
     // check the list of excluded files/dirs
-    if (is_filedir_excluded(relpath)==true)
+    if(is_oper_verify)
+    {
+        simul=true;
+    } else if (is_filedir_excluded(relpath)==true)
     {
         excluded=true;
+        simul=true;
     }
     else if (minorerr==false) // file not excluded and no error yet
     {
@@ -790,8 +826,9 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
         extractar_listing_print_file(exar, objtype, relpath);
     }
     
-    if ((minorerr==false) && (datafile_open_write(datafile, fullpath, excluded, sparse)<0))
-        minorerr=true;
+    if ((minorerr==false) && (datafile_open_write(datafile, fullpath, simul, sparse)<0))
+	if (excluded)
+	    minorerr=true;
     
     msgprintf(MSG_DEBUG2, "restore_obj_regfile_unique(file=%s, size=%lld)\n", relpath, (long long)filesize);
     for (filepos=0; (minorerr==false) && (filesize>0) && (filepos < filesize) && (get_interrupted()==false); filepos+=blkinfo.blkrealsize)
@@ -826,7 +863,7 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
     if ((minorerr==false) && (datafile_close(datafile, md5sumcalc, sizeof(md5sumcalc))!=0))
         minorerr=true;
     
-    if ((minorerr==false) && (excluded==false))
+    if ((minorerr==false) && (excluded==false) && !is_oper_verify)
     {
         if (extractar_restore_attr_everything(exar, objtype, fullpath, relpath, d)!=0)
         {   msgprintf(MSG_STACK, "cannot restore file attributes for file [%s]\n", relpath);
@@ -1329,6 +1366,76 @@ filesystem_extract_umount:
     return ret;
 }
 
+int extractar_filesystem_verify(cextractar *exar, cdico *dicofs, cstrdico *dicocmdline)
+{
+    char magic[FSA_SIZEOF_MAGIC+1];
+    cdico *dicobegin=NULL;
+    cdico *dicoend=NULL;
+    char mntbuf[PATH_MAX];
+    int errors=0;
+    u64 minver;
+    u64 curver;
+    int fstype;
+    int ret=0;
+
+    // init
+    memset(magic, 0, sizeof(magic));
+
+    // check that the minimum fsarchiver version required is ok
+    if (dico_get_u64(dicofs, 0, FSYSHEADKEY_MINFSAVERSION, &minver)!=0)
+        minver=FSA_VERSION_BUILD(0, 6, 4, 0); // fsarchiver-0.6.4 is the first fsa version having fileformat="FsArCh_002"
+    curver=FSA_VERSION_BUILD(PACKAGE_VERSION_A, PACKAGE_VERSION_B, PACKAGE_VERSION_C, PACKAGE_VERSION_D);
+    msgprintf(MSG_VERB2, "Current fsarchiver version: %d.%d.%d.%d\n", (int)FSA_VERSION_GET_A(curver),
+        (int)FSA_VERSION_GET_B(curver), (int)FSA_VERSION_GET_C(curver), (int)FSA_VERSION_GET_D(curver));
+    msgprintf(MSG_VERB2, "Minimum fsarchiver version for that filesystem: %d.%d.%d.%d\n", (int)FSA_VERSION_GET_A(minver), 
+        (int)FSA_VERSION_GET_B(minver), (int)FSA_VERSION_GET_C(minver), (int)FSA_VERSION_GET_D(minver));
+    if (curver < minver)
+    {   errprintf("This filesystem can only be restored with fsarchiver %d.%d.%d.%d or more recent\n",
+        (int)FSA_VERSION_GET_A(minver), (int)FSA_VERSION_GET_B(minver), (int)FSA_VERSION_GET_C(minver),
+        (int)FSA_VERSION_GET_D(minver));
+        return -1;
+    }
+
+    // ---- read filesystem-header from archive
+    if (queue_dequeue_header(&g_queue, &dicobegin, magic, NULL)<=0)
+    {   errprintf("queue_dequeue_header() failed: cannot read file system dico\n");
+        return -1;
+    }
+    dico_destroy(dicobegin);
+
+    if (memcmp(magic, FSA_MAGIC_FSYB, FSA_SIZEOF_MAGIC)!=0)
+    {   errprintf("header is not what we expected: found=[%s] and expected=[%s]\n", magic, FSA_MAGIC_FSYB);
+        return -1;
+    }
+
+    // Fixed to ext2
+    fstype=0;
+
+    if (extractar_extract_read_objects(exar, &errors, "dummy", fstype)!=0)
+    {   msgprintf(MSG_STACK, "extract_read_objects(%s) failed\n", mntbuf);
+        return -1;
+    }
+    else if (errors>0)
+    {   msgprintf(MSG_DEBUG1, "extract_read_objects(%s) worked with errors\n", mntbuf);
+        return -1;
+    }
+
+    // read "end of file-system" header from archive
+    if (queue_dequeue_header(&g_queue, &dicoend, magic, NULL)<=0)
+    {   errprintf("queue_dequeue_header() failed\n");
+        return -1;
+    }
+    dico_destroy(dicoend);
+
+    if ((get_interrupted()==false) && (memcmp(magic, FSA_MAGIC_DATF, FSA_SIZEOF_MAGIC)!=0))
+    {   errprintf("header is not what we expected: found=[%s] and expected=[%s]\n", magic, FSA_MAGIC_DATF);
+		return ret;
+    }
+
+    return ret;
+}
+
+
 int oper_restore(char *archive, int argc, char **argv, int oper)
 {
     cdico *dicofsinfo[FSA_MAX_FSPERARCH];
@@ -1383,6 +1490,7 @@ int oper_restore(char *archive, int argc, char **argv, int oper)
             break;
             
         case OPER_RESTDIR: // the files are all considered as belonging to fsid==0
+	case OPER_VERIFY:
             g_fsbitmap[0]=1;
             break;
     }
@@ -1425,7 +1533,7 @@ int oper_restore(char *archive, int argc, char **argv, int oper)
     }
     
     // show archive information if command is OPER_ARCHINFO
-    if (oper==OPER_ARCHINFO && archinfo_show_mainhead(&exar.ai, dicomainhead)!=0)
+    if ((oper==OPER_ARCHINFO || oper==OPER_VERIFY) && archinfo_show_mainhead(&exar.ai, dicomainhead)!=0)
     {   errprintf("archinfo_show_mainhead(%s) failed\n", archive);
         goto do_extract_error;
     }
@@ -1473,7 +1581,7 @@ int oper_restore(char *archive, int argc, char **argv, int oper)
             goto do_extract_error;
         }
         
-        if (oper==OPER_ARCHINFO && archinfo_show_fshead(dicofsinfo[i], i)!=0)
+        if ((oper==OPER_ARCHINFO  || oper==OPER_VERIFY) && archinfo_show_fshead(dicofsinfo[i], i)!=0)
         {   errprintf("archinfo_show_fshead() failed\n");
             goto do_extract_error;
         }
@@ -1515,8 +1623,8 @@ int oper_restore(char *archive, int argc, char **argv, int oper)
             // extract filesystem contents
             for (i=0; (i < exar.ai.fscount) && (i < FSA_MAX_FSPERARCH) && (get_abort()==false); i++)
             {
-                if (dicoargv[i]!=NULL) // that filesystem has been requested on the command line
-                {
+		 if (dicoargv[i]!=NULL) // that filesystem has been requested on the command line
+		 {
                     exar.fsid=i;
                     memset(&exar.stats, 0, sizeof(exar.stats)); // init stats to zero
                     msgprintf(MSG_VERB1, "============= extracting filesystem %d =============\n", i);
@@ -1566,8 +1674,45 @@ int oper_restore(char *archive, int argc, char **argv, int oper)
         {   errprintf("unsupported archtype: %d\n", exar.ai.archtype);
             goto do_extract_error;
         }
+    } else if(oper==OPER_VERIFY) {
+	is_oper_verify = true;
+
+        if (exar.ai.archtype==ARCHTYPE_FILESYSTEMS)
+        {
+            // extract filesystem contents
+            for (i=0; (i < exar.ai.fscount) && (i < FSA_MAX_FSPERARCH) && (get_abort()==false); i++)
+            {
+                    exar.fsid=i;
+                    memset(&exar.stats, 0, sizeof(exar.stats)); // init stats to zero
+                    msgprintf(MSG_VERB1, "============= verifying filesystem %d =============\n", i);
+                    if (extractar_filesystem_verify(&exar, dicofsinfo[i], dicoargv[i])!=0)
+                    {   msgprintf(MSG_STACK, "verify_filesystem(%d) failed\n", i);
+                        goto do_extract_error;
+                    }
+                    if (get_abort()==false)
+                        stats_show(exar.stats, i);
+                    totalerr+=stats_errcount(exar.stats);
+                }
+        }
+        else if (exar.ai.archtype==ARCHTYPE_DIRECTORIES)
+        {
+            exar.fsid=0;
+            destdir="dummy";
+            memset(&exar.stats, 0, sizeof(exar.stats)); // init stats to zero
+            if (extractar_extract_read_objects(&exar, &errors, destdir, 0)!=0) // TODO: get the right fstype
+            {   errprintf("extract_read_objects(%s) failed\n", destdir);
+                goto do_extract_error;
+            }
+            stats_show(exar.stats, 0);
+            totalerr+=stats_errcount(exar.stats);
+        }
+        else
+        {   errprintf("unsupported archtype: %d\n", exar.ai.archtype);
+            goto do_extract_error;
+        }
+
     }
-    
+
     if (get_abort()==true)
         msgprintf(MSG_FORCE, "operation aborted by user\n");
     
-- 
1.7.10.4



oliver
Posts: 3
Joined: Wed Jul 03, 2013 9:55 pm

Re: Add command to verify an archive

Post by oliver » Wed Jul 03, 2013 10:10 pm

And this is a patch for a verify on the latest git (on 6a7efc6). Also only basically tested, but seems to work.

Hope these patches can give a contribution to this nice tool :)!

A+, BR, Oliver

Code: Select all

From cc328050f90dbb3a82b8f0b338efc8984345d820 Mon Sep 17 00:00:00 2001
From: Oliver Winker <[email protected]>
Date: Wed, 3 Jul 2013 22:53:31 +0200
Subject: [PATCH] Add verify oper for ARCHTYPE_FILESYSTEMS and
 ARCHTYPE_DIRECTORIES

---
 src/fsarchiver.c |    8 ++
 src/fsarchiver.h |    3 +-
 src/restore.c    |  214 +++++++++++++++++++++++++++++++++++++++++++++---------
 3 files changed, 191 insertions(+), 34 deletions(-)

diff --git a/src/fsarchiver.c b/src/fsarchiver.c
index 3dbcdc8..3682ff1 100644
--- a/src/fsarchiver.c
+++ b/src/fsarchiver.c
@@ -76,6 +76,7 @@ void usage(char *progname, bool examples)
     msgprintf(MSG_FORCE, " * showpt: show partition tables which have been save in an archive\n");
     msgprintf(MSG_FORCE, " * archinfo: show information about an existing archive file and its contents\n");
     msgprintf(MSG_FORCE, " * probe [-v]: show list of local disks and filesystems\n");
+    msgprintf(MSG_FORCE, " * verify: verify an archive\n");
     msgprintf(MSG_FORCE, "<options>\n");
     msgprintf(MSG_FORCE, " -o: overwrite the archive if it already exists instead of failing\n");
     msgprintf(MSG_FORCE, " -v: verbose mode (can be used several times to increase the level of details)\n");
@@ -361,6 +362,11 @@ int process_cmdline(int argc, char **argv)
         runasroot=false;
         argcok=(argc==1);
     }
+    else if (strcmp(command, "verify")==0)
+    {   cmd=OPER_VERIFY;
+        runasroot=false;
+        argcok=(argc<=1);
+    }
     else // command not found
     {
         errprintf("[%s] is not a valid command.\n", command);
@@ -428,6 +434,7 @@ int process_cmdline(int argc, char **argv)
         case OPER_SAVEPT:
         case OPER_RESTPT:
         case OPER_SHOWPT:
+        case OPER_VERIFY:
             snprintf(g_archive, PATH_MAX, *argv);
             argv++;
             argc--;
@@ -472,6 +479,7 @@ int process_cmdline(int argc, char **argv)
         case OPER_ARCHINFO:
         case OPER_RESTPT:
         case OPER_SHOWPT:
+	case OPER_VERIFY:
             ret=restore(argc, argv, cmd);
             break;
         case OPER_PROBE:
diff --git a/src/fsarchiver.h b/src/fsarchiver.h
index c1320f1..fd897a4 100644
--- a/src/fsarchiver.h
+++ b/src/fsarchiver.h
@@ -31,7 +31,8 @@
 
 // ----------------------------------- fsarchiver commands ------------------------------------------
 enum {OPER_NULL=0, OPER_SAVEFS, OPER_RESTFS, OPER_SAVEDIR, OPER_RESTDIR, 
-      OPER_ARCHINFO, OPER_PROBE, OPER_SAVEPT, OPER_RESTPT, OPER_SHOWPT};
+      OPER_ARCHINFO, OPER_PROBE, OPER_SAVEPT, OPER_RESTPT, OPER_SHOWPT,
+      OPER_VERIFY};
 
 // ----------------------------------- fsarchiver archive format ------------------------------------
 enum {FSA_FMT_NULL=0, FSA_FMT_06=1, FSA_FMT_07=2};
diff --git a/src/restore.c b/src/restore.c
index da557c2..6b7e5f4 100644
--- a/src/restore.c
+++ b/src/restore.c
@@ -66,6 +66,8 @@ typedef struct s_extractar
 }
 cextractar;
 
+static int is_oper_verify = false;
+
 // returns true if this file of a parent directory has been excluded
 int is_filedir_excluded(char *relpath)
 {
@@ -338,7 +340,10 @@ int extractar_restore_attr_std(cextractar *exar, u32 objtype, char *fullpath, ch
 int extractar_restore_attr_everything(cextractar *exar, int objtype, char *fullpath, char *relpath, cdico *dicoattr)
 {
     int res=0;
-    
+
+    if (is_oper_verify)
+	goto extractar_restore_attr_everything_done;
+
     // ---- restore standard attributes
     res+=extractar_restore_attr_std(exar, objtype, fullpath, relpath, dicoattr);
     
@@ -347,7 +352,8 @@ int extractar_restore_attr_everything(cextractar *exar, int objtype, char *fullp
     
     // ---- restore windows attributes
     res+=extractar_restore_attr_windows(exar, objtype, fullpath, relpath, dicoattr);
-    
+
+extractar_restore_attr_everything_done:
     return (res==0)?(0):(-1);
 }
 
@@ -361,6 +367,9 @@ int extractar_restore_obj_symlink(cextractar *exar, char *fullpath, char *relpat
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
+
+    if (is_oper_verify)
+	goto extractar_restore_obj_symlink_done;
     
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -434,7 +443,8 @@ int extractar_restore_obj_symlink(cextractar *exar, char *fullpath, char *relpat
         sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_symlink_err;
     }
-    
+
+extractar_restore_obj_symlink_done:
     dico_destroy(d);
     exar->stats.cnt_symlink++;
     return 0; // success
@@ -455,6 +465,9 @@ int extractar_restore_obj_hardlink(cextractar *exar, char *fullpath, char *relpa
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
+
+    if (is_oper_verify)
+	goto extractar_restore_obj_hardlink_done;
     
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -496,7 +509,8 @@ int extractar_restore_obj_hardlink(cextractar *exar, char *fullpath, char *relpa
         sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_hardlink_err;
     }
-    
+
+extractar_restore_obj_hardlink_done:
     dico_destroy(d);
     exar->stats.cnt_hardlink++;
     return 0; // success
@@ -516,6 +530,9 @@ int extractar_restore_obj_devfile(cextractar *exar, char *fullpath, char *relpat
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
+
+    if (is_oper_verify)
+	goto extractar_restore_obj_devfile_done;
     
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -552,7 +569,8 @@ int extractar_restore_obj_devfile(cextractar *exar, char *fullpath, char *relpat
         sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_devfile_err;
     }
-    
+
+extractar_restore_obj_devfile_done:
     dico_destroy(d);
     exar->stats.cnt_special++;
     return 0; // success
@@ -570,7 +588,10 @@ int extractar_restore_obj_directory(cextractar *exar, char *fullpath, char *relp
     
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
-    
+
+   if (is_oper_verify)
+	goto extractar_restore_obj_directory_done;
+
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
         goto extractar_restore_obj_directory_err;
@@ -599,7 +620,8 @@ int extractar_restore_obj_directory(cextractar *exar, char *fullpath, char *relp
         sysprintf("utimes(%s) failed\n", parentdir);
         goto extractar_restore_obj_directory_err;
     }
-    
+
+extractar_restore_obj_directory_done:
     dico_destroy(d);
     exar->stats.cnt_dir++;
     return 0; // success
@@ -632,13 +654,14 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
     s64 lres;
     int res;
     int i;
+    int simul = false;
     
     // init
     errors=0;
     memset(&blkinfo, 0, sizeof(blkinfo));
     regmulti_init(&regmulti, FSA_MAX_BLKSIZE);
     datafile=datafile_alloc();
-    
+
     // ---- dequeue header for each small file which is part of that group
     if (dico_get_u32(dicofirstfile, 0, DISKITEMKEY_MULTIFILESCOUNT, &filescount)!=0)
     {
@@ -716,17 +739,21 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
         exar->cost_current+=datsize; // filesize
         
         // check the list of excluded files/dirs
-        if (is_filedir_excluded(relpath)!=true)
+        if (is_filedir_excluded(relpath)!=true || is_oper_verify)
         {
-            // create parent directory if necessary
-            extract_dirpath(fullpath, parentdir, sizeof(parentdir));
-            mkdir_recursive(parentdir);
-            
-            // backup parent dir atime/mtime
-            get_parent_dir_time_attrib(fullpath, parentdir, sizeof(parentdir), tv);
-            
-            extractar_listing_print_file(exar, tmpobjtype, relpath);
-            
+	    if (is_oper_verify) {
+		simul = true;
+	    } else {
+		// create parent directory if necessary
+		extract_dirpath(fullpath, parentdir, sizeof(parentdir));
+		mkdir_recursive(parentdir);
+
+		// backup parent dir atime/mtime
+		get_parent_dir_time_attrib(fullpath, parentdir, sizeof(parentdir), tv);
+            }
+
+	    extractar_listing_print_file(exar, tmpobjtype, relpath);
+
             if (dico_get_data(filehead, DICO_OBJ_SECTION_STDATTR, DISKITEMKEY_MD5SUM, md5sumorig, 16, NULL))
             {
                 errprintf("cannot get md5sum from file footer for file=[%s]\n", relpath);
@@ -734,7 +761,7 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
                 goto extractar_restore_obj_regfile_multi_err;
             }
             
-            if (datafile_open_write(datafile, fullpath, false, false)<0)
+            if (datafile_open_write(datafile, fullpath, simul, false)<0)
                 goto extractar_restore_obj_regfile_multi_err;
             
             res=datafile_write(datafile, databuf, datsize);
@@ -762,7 +789,7 @@ int extractar_restore_obj_regfile_multi(cextractar *exar, char *destdir, cdico *
             }
             
             // restore parent dir mtime/atime
-            if (utimes(parentdir, tv)!=0)
+            if (!is_oper_verify && (utimes(parentdir, tv)!=0))
             {
                 sysprintf("utimes(%s) failed\n", parentdir);
                 goto extractar_restore_obj_regfile_multi_err;
@@ -802,11 +829,12 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
     u64 filepos=0;
     u64 flags=0;
     s64 lres;
+    int simul=false;
     
     // init
     memset(&blkinfo, 0, sizeof(blkinfo));
     datafile=datafile_alloc();
-    
+
     if (dico_get_u64(d, DICO_OBJ_SECTION_STDATTR, DISKITEMKEY_SIZE, &filesize)!=0)
     {
         errprintf("Cannot read filesize DISKITEMKEY_SIZE from archive for file=[%s]\n", relpath);
@@ -820,9 +848,13 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
     exar->cost_current+=filesize;
     
     // check the list of excluded files/dirs
-    if (is_filedir_excluded(relpath)==true)
+    if(is_oper_verify)
+    {
+	simul = true;
+    } else if (is_filedir_excluded(relpath)==true)
     {
         excluded=true;
+	simul = true;
     }
     else if (minorerr==false) // file not excluded and no error yet
     {
@@ -837,9 +869,10 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
         extractar_listing_print_file(exar, objtype, relpath);
     }
     
-    if ((minorerr==false) && (datafile_open_write(datafile, fullpath, excluded, sparse)<0))
-        minorerr=true;
-    
+    if ((minorerr==false) && (datafile_open_write(datafile, fullpath, simul, sparse)<0))
+	if (excluded)
+	    minorerr=true;
+
     msgprintf(MSG_DEBUG2, "restore_obj_regfile_unique(file=%s, size=%lld)\n", relpath, (long long)filesize);
     for (filepos=0; (minorerr == false) && (filesize > 0) && (filepos < filesize) && (get_status() == STATUS_RUNNING); filepos+=blkinfo.blkrealsize)
     {
@@ -885,7 +918,7 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
         }
     
         // restore parent dir mtime/atime
-        if (utimes(parentdir, tv)!=0)
+        if (!is_oper_verify && (utimes(parentdir, tv)!=0))
         {
             sysprintf("utimes(%s) failed\n", parentdir);
             minorerr=true;
@@ -1077,7 +1110,7 @@ int extractar_extract_read_objects(cextractar *exar, int *errors, char *destdir,
                 errprintf("queue_dequeue_header() failed\n");
                 (*errors)++;
             }
-            
+
             if (checkfsid == exar->fsid) // if filesystem-id is correct
             {
                 if ((res = extractar_restore_object(exar, &curerr, destdir, dicoattr, fstype)) != 0)
@@ -1456,6 +1489,76 @@ filesystem_extract_umount:
     return ret;
 }
 
+int extractar_filesystem_verify(cextractar *exar, cdico *dicofs, cstrdico *dicocmdline)
+{
+    cdico *dicobegin=NULL;
+    cdico *dicoend=NULL;
+    char mntbuf[PATH_MAX];
+    u32 headertype;
+    int errors=0;
+    u64 minver;
+    u64 curver;
+    int fstype;
+    int ret=0;
+
+    // check that the minimum fsarchiver version required is ok
+    if (dico_get_u64(dicofs, 0, FSYSHEADKEY_MINFSAVERSION, &minver) != 0)
+    {
+        errprintf("dico_get_u64(FSYSHEADKEY_MINFSAVERSION) failed\n");
+        return -1;
+    }
+    curver=FSA_VERSION_BUILD(PACKAGE_VERSION_A, PACKAGE_VERSION_B, PACKAGE_VERSION_C, PACKAGE_VERSION_D);
+    msgprintf(MSG_VERB2, "Current fsarchiver version: %d.%d.%d.%d\n", (int)FSA_VERSION_GET_A(curver),
+        (int)FSA_VERSION_GET_B(curver), (int)FSA_VERSION_GET_C(curver), (int)FSA_VERSION_GET_D(curver));
+    msgprintf(MSG_VERB2, "Minimum fsarchiver version for that filesystem: %d.%d.%d.%d\n", (int)FSA_VERSION_GET_A(minver),
+        (int)FSA_VERSION_GET_B(minver), (int)FSA_VERSION_GET_C(minver), (int)FSA_VERSION_GET_D(minver));
+    if (curver < minver)
+    {
+        errprintf("This filesystem can only be restored with fsarchiver %d.%d.%d.%d or more recent\n",
+        (int)FSA_VERSION_GET_A(minver), (int)FSA_VERSION_GET_B(minver), (int)FSA_VERSION_GET_C(minver),
+        (int)FSA_VERSION_GET_D(minver));
+        return -1;
+    }
+
+    // read filesystem-header from archive
+    if (queue_dequeue_header(g_queue, &dicobegin, &headertype, NULL) <= 0)
+    {
+        errprintf("queue_dequeue_header() failed: cannot read file system dico\n");
+        return -1;
+    }
+    dico_destroy(dicobegin);
+
+    // Fixed to ext2
+    fstype=0;
+
+    if (extractar_extract_read_objects(exar, &errors, "dummy", fstype) != 0)
+    {
+        msgprintf(MSG_STACK, "extract_read_objects(%s) failed\n", mntbuf);
+        return -1;
+    }
+    else if (errors > 0)
+    {
+        msgprintf(MSG_DEBUG1, "extract_read_objects(%s) worked with errors\n", mntbuf);
+        return -1;
+    }
+
+    // read "end of file-system" header from archive
+    if (queue_dequeue_header(g_queue, &dicoend, &headertype, NULL) <= 0)
+    {
+        errprintf("queue_dequeue_header() failed\n");
+        return -1;
+    }
+    dico_destroy(dicoend);
+
+    if ((get_status() == STATUS_RUNNING) && (headertype != FSA_HEADTYPE_DATF))
+    {
+        errprintf("header is not what we expected: found=[%ld] and expected=[%ld]\n", (long)headertype, (long)FSA_HEADTYPE_DATF);
+        return -1;
+    }
+
+    return ret;
+}
+
 int restore(int argc, char **argv, int oper)
 {
     pthread_t thread_decomp[FSA_MAX_COMPJOBS]; // reads blocks from queue and does decompression/decryption
@@ -1533,6 +1636,7 @@ int restore(int argc, char **argv, int oper)
             break;
 
        case OPER_RESTDIR: // the files are all considered as belonging to fsid==0
+       case OPER_VERIFY:
             g_fsbitmap[0]=1;
             break;
 
@@ -1617,7 +1721,8 @@ int restore(int argc, char **argv, int oper)
     switch (archinfo.archtype)
     {
         case ARCHTYPE_FILESYSTEMS:
-            if (oper != OPER_RESTFS && oper != OPER_RESTPT && oper != OPER_SHOWPT && oper != OPER_ARCHINFO)
+            if (oper != OPER_RESTFS && oper != OPER_RESTPT && oper != OPER_SHOWPT && oper != OPER_ARCHINFO
+		&& oper != OPER_VERIFY)
             {
                 errprintf("this archive contains filesystems. The command is not appropriate for that type of archive\n");
                 goto do_extract_error;
@@ -1625,7 +1730,7 @@ int restore(int argc, char **argv, int oper)
             break;
 
         case ARCHTYPE_DIRECTORIES:
-            if (oper != OPER_RESTDIR && oper != OPER_ARCHINFO)
+            if (oper != OPER_RESTDIR && oper != OPER_ARCHINFO && oper != OPER_VERIFY)
             {
                 errprintf("this archive contains flat files & directories. The command is not appropriate for that type of archive\n");
                 goto do_extract_error;
@@ -1822,6 +1927,9 @@ int restore(int argc, char **argv, int oper)
                 set_status(STATUS_FINISHED, "");
             break;
 
+        case OPER_VERIFY:
+	    is_oper_verify = true;
+	    // fall through, to show arch info
         case OPER_ARCHINFO:
             if (archinfo_show_mainhead(&archinfo)!=0)
             {
@@ -1836,10 +1944,50 @@ int restore(int argc, char **argv, int oper)
                     goto do_extract_error;
                 }
             }
-            // stop reading the archive file
-            if (get_status() == STATUS_RUNNING)
-                set_status(STATUS_FINISHED, "");
-            break;
+
+	    if (!is_oper_verify) {
+		// stop reading the archive file
+		if (get_status() == STATUS_RUNNING)
+		    set_status(STATUS_FINISHED, "");
+		break;
+	    }
+
+	    // actual OPER_VERIFY code
+	    switch(archinfo.archtype) {
+		case ARCHTYPE_FILESYSTEMS:
+		   for (i=0; (i < archinfo.fscount) && (i < FSA_MAX_FSPERARCH) && (get_status() == STATUS_RUNNING); i++)
+		    {
+			exar.fsid=i;
+			memset(&exar.stats, 0, sizeof(exar.stats)); // init stats to zero
+			msgprintf(MSG_VERB1, "============= verifying filesystem %d =============\n", i);
+			if (extractar_filesystem_verify(&exar, dicofsinfo[i], dicoargv[i])!=0)
+			{
+			    msgprintf(MSG_STACK, "verify_filesystem(%d) failed\n", i);
+			    goto do_extract_error;
+			}
+			if (get_status() == STATUS_RUNNING)
+			    stats_show(exar.stats, i);
+			totalerr+=stats_errcount(exar.stats);
+		    }
+		break;
+		case ARCHTYPE_DIRECTORIES:
+		    exar.fsid=0;
+		    memset(&exar.stats, 0, sizeof(exar.stats)); // init stats to zero
+		    if (extractar_extract_read_objects(&exar, &errors, "dummy", 0)!=0) // TODO: get the right fstype
+		    {
+			errprintf("extract_read_objects(%s) failed\n", "dummy");
+			goto do_extract_error;
+		    }
+		    stats_show(exar.stats, 0);
+		    totalerr += stats_errcount(exar.stats);
+		break;
+
+		default:
+		    errprintf("OPER_VERIFY unsupported\n");
+		    goto do_extract_error;
+	    }
+	    break;
+
     }
 
     switch (get_status())
-- 
1.7.10.4



oliver
Posts: 3
Joined: Wed Jul 03, 2013 9:55 pm

Re: Add command to verify an archive

Post by oliver » Thu Jul 04, 2013 7:50 pm

Hi, here still another two patches:

- 0002-Fix-logging-of-verify-oper.fsarchiver-0.6.17.patch
. related to the previous verify oper patch on fsarchiver-0.6.17

- 0003-Fix-valgrind-drd-findings.fsarchiver-0.6.17.patch

I post them for the time being just for fsarchiver-0.6.17. In case there is interest also in the patches for the git head, just tell me.

BR, A+, Oliver ;)

Code: Select all

From 5009703b1a77c56e6d839edf45a8e5dc66a8ca90 Mon Sep 17 00:00:00 2001
From: Oliver Winker <[email protected]>
Date: Thu, 4 Jul 2013 21:39:19 +0200
Subject: [PATCH 2/3] Fix logging of verify oper

---
 src/oper_restore.c |   23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/src/oper_restore.c b/src/oper_restore.c
index a5e8896..2f99322 100644
--- a/src/oper_restore.c
+++ b/src/oper_restore.c
@@ -351,8 +351,10 @@ int extractar_restore_obj_symlink(cextractar *exar, char *fullpath, char *relpat
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
 
-    if (is_oper_verify)
+    if (is_oper_verify) {
+	extractar_listing_print_file(exar, objtype, relpath);
 	goto extractar_restore_obj_symlink_done;
+    }
 
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -443,8 +445,10 @@ int extractar_restore_obj_hardlink(cextractar *exar, char *fullpath, char *relpa
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
 
-    if (is_oper_verify)
+    if (is_oper_verify) {
+	extractar_listing_print_file(exar, objtype, relpath);
 	goto extractar_restore_obj_hardlink_done;
+    }
 
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -504,8 +508,10 @@ int extractar_restore_obj_devfile(cextractar *exar, char *fullpath, char *relpat
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
 
-    if (is_oper_verify)
+    if (is_oper_verify) {
+	extractar_listing_print_file(exar, objtype, relpath);
 	goto extractar_restore_obj_devfile_done;
+    }
     
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -559,8 +565,10 @@ int extractar_restore_obj_directory(cextractar *exar, char *fullpath, char *relp
     // update cost statistics and progress bar
     exar->cost_current+=FSA_COST_PER_FILE; 
 
-    if (is_oper_verify)
+    if (is_oper_verify) {
+	extractar_listing_print_file(exar, objtype, relpath);
 	goto extractar_restore_obj_directory_done;
+    }
     
     // check the list of excluded files/dirs
     if (is_filedir_excluded(relpath)==true)
@@ -821,10 +829,9 @@ int extractar_restore_obj_regfile_unique(cextractar *exar, char *fullpath, char
         
         // backup parent dir atime/mtime
         get_parent_dir_time_attrib(fullpath, parentdir, sizeof(parentdir), tv);
-        
-        // show progress bar
-        extractar_listing_print_file(exar, objtype, relpath);
     }
+    // show progress bar
+    extractar_listing_print_file(exar, objtype, relpath);
     
     if ((minorerr==false) && (datafile_open_write(datafile, fullpath, simul, sparse)<0))
 	if (excluded)
@@ -1587,7 +1594,7 @@ int oper_restore(char *archive, int argc, char **argv, int oper)
         }
         
         // calculate total cost of the restfs
-        if ((dicoargv[i]!=NULL) && (dico_get_u64(dicofsinfo[i], 0, FSYSHEADKEY_TOTALCOST, &fscost)==0))
+        if ((dicoargv[i]!=NULL || oper==OPER_VERIFY) && (dico_get_u64(dicofsinfo[i], 0, FSYSHEADKEY_TOTALCOST, &fscost)==0))
             exar.cost_global+=fscost;
     }
     
-- 
1.7.10.4


Code: Select all

From 7fb7aca1fbf411ca91d2313e1152518f65a2e1b4 Mon Sep 17 00:00:00 2001
From: Oliver Winker <[email protected]>
Date: Thu, 4 Jul 2013 21:43:22 +0200
Subject: [PATCH 3/3] Fix valgrind drd findings

Fix following findings by "valgrind --trace-children=yes --tool=drd"
---
==16871== Thread 6:
==16871== Probably a race condition: condition variable 0x6362f0 has been signaled but the associated mutex 0x6362c8 is not locked by the signalling thread.
==16871==    at 0x4C33602: [email protected]* (drd_pthread_intercepts.c:818)
==16871==    by 0x420B7D: queue_add_header_internal (queue.c:310)
==16871==    by 0x420CB5: queue_add_header (queue.c:258)
==16871==    by 0x413AD8: thread_reader_fct (thread_archio.c:166)
==16871==    by 0x4C2E364: vgDrd_thread_wrapper (drd_pthread_intercepts.c:355)
==16871==    by 0x504BE0D: start_thread (pthread_create.c:311)
==16871==    by 0x6AB295C: clone (clone.S:113)
[...]
---
---
 src/queue.c |   28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/src/queue.c b/src/queue.c
index a56d6d7..c542fc3 100644
--- a/src/queue.c
+++ b/src/queue.c
@@ -113,8 +113,8 @@ s64 queue_set_end_of_queue(cqueue *q, bool state)
 
     assert(pthread_mutex_lock(&q->mutex)==0);
     q->endofqueue=state;
-    assert(pthread_mutex_unlock(&q->mutex)==0);
     pthread_cond_broadcast(&q->cond);
+    assert(pthread_mutex_unlock(&q->mutex)==0);
     return FSAERR_SUCCESS;
 }
 
@@ -235,8 +235,8 @@ s64 queue_add_block(cqueue *q, cblockinfo *blkinfo, int status)
     q->itemcount++;
     item->itemnum=q->curitemnum++;
     
-    assert(pthread_mutex_unlock(&q->mutex)==0);
     pthread_cond_broadcast(&q->cond);
+    assert(pthread_mutex_unlock(&q->mutex)==0);
     
     return FSAERR_SUCCESS;
 }
@@ -306,8 +306,8 @@ s64 queue_add_header_internal(cqueue *q, cheadinfo *headinfo)
     }
     
     q->itemcount++;
-    assert(pthread_mutex_unlock(&q->mutex)==0);
     pthread_cond_broadcast(&q->cond);
+    assert(pthread_mutex_unlock(&q->mutex)==0);
     
     return FSAERR_SUCCESS;
 }
@@ -336,8 +336,8 @@ s64 queue_replace_block(cqueue *q, s64 itemnum, cblockinfo *blkinfo, int newstat
         {
             cur->status=newstatus;
             cur->blkinfo=*blkinfo;
-            assert(pthread_mutex_unlock(&q->mutex)==0);
             pthread_cond_broadcast(&q->cond);
+            assert(pthread_mutex_unlock(&q->mutex)==0);
             return FSAERR_SUCCESS;
         }
     }
@@ -393,8 +393,8 @@ s64 queue_get_first_block_todo(cqueue *q, cblockinfo *blkinfo)
                 *blkinfo=cur->blkinfo;
                 cur->status=QITEM_STATUS_PROGRESS;
                 itemfound=cur->itemnum;
-                assert(pthread_mutex_unlock(&q->mutex)==0);
                 pthread_cond_broadcast(&q->cond);
+                assert(pthread_mutex_unlock(&q->mutex)==0);
                 return itemfound; // ">0" means item found
             }
         }
@@ -444,8 +444,8 @@ s64 queue_dequeue_first(cqueue *q, int *type, cheadinfo *headinfo, cblockinfo *b
                 q->head=cur->next;
                 free(cur);
                 q->itemcount--;
-                assert(pthread_mutex_unlock(&q->mutex)==0);
                 pthread_cond_broadcast(&q->cond);
+                assert(pthread_mutex_unlock(&q->mutex)==0);
                 return itemfound; // ">0" means item found
             }
             else if (cur->type==QITEM_TYPE_HEADER) // item to dequeue is a dico
@@ -456,8 +456,8 @@ s64 queue_dequeue_first(cqueue *q, int *type, cheadinfo *headinfo, cblockinfo *b
                 q->head=cur->next;
                 free(cur);
                 q->itemcount--;
-                assert(pthread_mutex_unlock(&q->mutex)==0);
                 pthread_cond_broadcast(&q->cond);
+                assert(pthread_mutex_unlock(&q->mutex)==0);
                 return itemfound; // ">0" means item found
             }
             else
@@ -517,15 +517,15 @@ s64 queue_dequeue_block(cqueue *q, cblockinfo *blkinfo)
         free(cur);
         q->blkcount--;
         q->itemcount--;
-        assert(pthread_mutex_unlock(&q->mutex)==0);
         pthread_cond_broadcast(&q->cond);
+        assert(pthread_mutex_unlock(&q->mutex)==0);
         return itemnum;
     }
     else
     {
         errprintf("dequeue - wrong type of data in the queue: wanted a block, found an header\n");
-        assert(pthread_mutex_unlock(&q->mutex)==0);
         pthread_cond_broadcast(&q->cond);
+        assert(pthread_mutex_unlock(&q->mutex)==0);
         return FSAERR_WRONGTYPE;  // ok but not found
     }
 }
@@ -589,18 +589,18 @@ s64 queue_dequeue_header_internal(cqueue *q, cheadinfo *headinfo)
             itemnum=cur->itemnum;
             free(cur);
             q->itemcount--;
-            assert(pthread_mutex_unlock(&q->mutex)==0);
             pthread_cond_broadcast(&q->cond);
+            assert(pthread_mutex_unlock(&q->mutex)==0);
             return itemnum;
         case QITEM_TYPE_BLOCK:
             errprintf("dequeue - wrong type of data in the queue: expected a dico and found a block\n");
-            assert(pthread_mutex_unlock(&q->mutex)==0);
             pthread_cond_broadcast(&q->cond);
+            assert(pthread_mutex_unlock(&q->mutex)==0);
             return FSAERR_WRONGTYPE;  // ok but not found
         default: // should never happen
             errprintf("dequeue - wrong type of data in the queue: expected a dico and found an unknown item\n");
-            assert(pthread_mutex_unlock(&q->mutex)==0);
             pthread_cond_broadcast(&q->cond);
+            assert(pthread_mutex_unlock(&q->mutex)==0);
             return FSAERR_WRONGTYPE;  // ok but not found
     }
 }
@@ -678,8 +678,8 @@ s64 queue_check_next_item(cqueue *q, int *type, char *magic)
         }
     }
     
-    assert(pthread_mutex_unlock(&q->mutex)==0);
     pthread_cond_broadcast(&q->cond);
+    assert(pthread_mutex_unlock(&q->mutex)==0);
     
     return FSAERR_ENOENT;  // not found
 }
@@ -725,7 +725,7 @@ s64 queue_destroy_first_item(cqueue *q)
     q->head=cur->next;
     free(cur);
     q->itemcount--;
-    assert(pthread_mutex_unlock(&q->mutex)==0);
     pthread_cond_broadcast(&q->cond);
+    assert(pthread_mutex_unlock(&q->mutex)==0);
     return FSAERR_SUCCESS;
 }
-- 
1.7.10.4



Lazy_Kent
Posts: 53
Joined: Sun Nov 15, 2009 1:08 pm
Location: Moscow, Russia

Re: Add command to verify an archive

Post by Lazy_Kent » Sun Jul 14, 2013 4:44 am

Thanks for contribution!
Haven't tested yet myself. I've forwarded your message to SystemRescueCd forum:
http://www.sysresccd.org/forums/viewtop ... =23&t=5132

Post Reply