ShareFile PowerShell and Windows Task Scheduler for local backup

  • 1
  • Problem
  • Updated 2 months ago
We are trying to accomplish backing up our entire ShareFile structure locally to a NAS. I have this currently setup and running. Only problem is, the task seems to be sporadic on how long it runs. Physically looking over the folder structure and files, I'm seeing that it's not finishing the sync before "Task Completed" as the Task Scheduler History shows. Sometimes it'll run just for a few minutes, other times it'll run for over 24 hours.

From what I can tell, everything is configured correctly, and I've also tested small syncs with success. The task is being ran with a domain account and proper permissions to the NAS for that account. The ShareFile account used is a Super User. Just for further information, this structure is over 2TB.

The Script

Add-PSSnapIn ShareFile
$sfLogin = Get-SfClient -Name "C:\ShareFileBackup\sflogin.sfps"
New-PSDrive -Name SFBackup -PSProvider ShareFile -Root "/" -Client $sfLogin
Sync-SfItem -ShareFilePath "/" -Synchronize -Download -LocalPath "\\wattnas\ShareFile" -Recursive -Strict

The Scheduled Task







Regards,
Barrett

Photo of Barrett Cowan

Barrett Cowan

  • 2 Posts
  • 0 Reply Likes

Posted 1 year ago

  • 1
Photo of Barrett Cowan

Barrett Cowan

  • 2 Posts
  • 0 Reply Likes
Still seeing the same behavior here. Anyone have any input? I have a feeling this may be an issue with the size of the entire structure and the -Strict option possibly. Is there anyway to log, I'm not seeing any options for it?
Photo of Peter Oven

Peter Oven

  • 22 Posts
  • 7 Reply Likes
With 2 TB I am not surprised that it's not completing the sync. I have 500 GB or so and I had to break the sync into sections, so that a failure of one part does not cause the whole thing to fail. I just run this every week using a scheduled task.


# Sharefile Backup Script
# Peter Oven
# 20180404 V1.0
# 20190717 V2.0
# 20190724 V2.1
#
# developed using Powershell SDK v1.92
# Based on code found at the following web pages
# https://github.com/citrix/ShareFile-PowerShell/releases
# https://support.citrix.com/article/CTX207460
# https://www.citrix.com/blogs/2014/05/16/getting-started-with-the-powershell-sdk/
# https://www.citrix.com/blogs/2014/11/13/transitioning-from-sfcli-to-sharefile-powershell/
# https://www.vexasoft.com/blogs/powershell/7255220-powershell-tutorial-try-catch-finally-and-error-handling-in-powershell

# Directories to Back Up
$sharedFoldersLocations = "External","WES Energy and Environment"
$wesLocations = "Individual Storage","WES Business","USDA","Reference Materials","Projects","AutoCAD Resources","BoilerMACT Compliance","Miscellaneous","Contracts","Proposals","Project Cost Estimating","USDA Technical Support"

# for manual adjustments to limit the backup extent
#$sharedFoldersLocations = $null
#$wesLocations = "BoilerMACT Compliance"

# logging system https://stackoverflow.com/questions/7834656/create-log-file-in-powershell
$logfile = "C:\SF\logs\sf-Backup.log"
Function Write-Log {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$False)]
[ValidateSet("INFO","WARN","ERROR","FATAL","DEBUG")]
[String]
$Level = "INFO",
[Parameter(Mandatory=$True)]
[string]
$Message,
[Parameter(Mandatory=$False)]
[string]
$logfile
)
$Stamp = (Get-Date).toString("yyyy/MM/dd HH:mm:ss")
$Line = "$Stamp $Level $Message"
If($logfile) {
Add-Content $logfile -Value $Line
}
Else {
Write-Output $Line
}
}

Function doSync {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True)]
[string]
$loc,
[Parameter(Mandatory=$True)]
[string]
$pathPrepend,
[Parameter(Mandatory=$True)]
[string]
$logfile
)
Add-PSSnapin ShareFile
# load login token
$sfc = Get-SfClient –Name "c:\SF\sflogin-poven.sfps"
# set up temporary sharefile drive
New-PSDrive -Name sfdrive -PSProvider ShareFile -Root "/" -Client $sfc
Write-Log "INFO" ("Starting "+$loc) $logfile
Try {
# Strict will delete locally any files which have been deleted on the sharefile account
#Sync-SfItem -ShareFilePath ("/Shared Folders"+$pathPrepend+$loc+"/") -Synchronize -Download -LocalPath ("c:\SF\Shared Folders"+$pathPrepend) -Recursive -Strict -ErrorAction Stop
Sync-SfItem -ShareFilePath ("/Shared Folders"+$pathPrepend+$loc+"/") -Synchronize -Download -LocalPath ("c:\SF\Shared Folders"+$pathPrepend) -Recursive -ErrorAction Stop
Write-Log "INFO" ("Backup of "+$loc+" complete") $logfile
}
Catch {
Write-Log "ERROR" ("Backup of "+$loc+" failed") $logfile
Write-Log "ERROR" $_.Exception.Message $logfile
}
Finally {
Remove-PSDrive sfdrive
Remove-PSSnapin ShareFile
}
}

foreach($location in $sharedFoldersLocations)
{
doSync $location "/" $logfile
}

foreach($location in $wesLocations)
{
doSync $location "/WES/" $logfile
}



Photo of Peter Oven

Peter Oven

  • 22 Posts
  • 7 Reply Likes
You have a good point that these should not be needed except once at the beginning and once at the end. I intentionally put them inside the function that runs over and over, in order to fully "clean up" and "start fresh" with each directory that I am backing up. My intention was that this would add reliability to the script since if an unknown Citrix error happens during one backup, that could potentially corrupt the structures or whatever in the "sfdrive", that directory backup might fail, but then my script will close out that dfdrive and remove the snapin, and then start fresh on the next directory in the sequence with reloading the snapin and creating a new (identical) sfdrive.

If this was a script that was being called in realtime by a  user's actions, then maybe it would be worth optimizing, but the extra adding and cleanup of the snapin and sfdrive in this case is negligible and the entire process runs in the background on a computer dedicated only to this task.

I have basically no error handling from the Citrix command other than fatal errors. If Citrix developed better error handling in their snapin, or a logging feature that worked, maybe I would do it differently.
Photo of ShareFile Admin VA

ShareFile Admin VA

  • 60 Posts
  • 10 Reply Likes
Thanks again.  We figured as much but wanted to confirm.
Photo of dan becker

dan becker

  • 2 Posts
  • 0 Reply Likes
Is there any way to exclude the SF folder name when it's copied to the local directory? See below snippet.

SF structure:
sfdirectory/project2/sub1
sfdirectory/project2/sub2
sfdirectory/project2/...
....
$sharedFoldersLocations = @{
    "sfdirectory/project2" = "C:\backup\content";
    "sfdirectory/project3" = "Z:\backup\new"
}

foreach($key in $sharedFoldersLocations.keys){
    doSync $key $sharedFoldersLocations.$key "/" $logfile
}

This results in a new directory being created locally, like so:
C:\backup\content\project2

I'd like it to copy like this:
C:\backup\content\sub1
C:\backup\content\sub2

I've tried every possible switch in Sync-SfItem to no avail, thanks!



Photo of Peter Oven

Peter Oven

  • 22 Posts
  • 7 Reply Likes
Interesting idea, you might need to add some code to search through the project2 directory and populate the $sharedFoldersLocations automatically, so that it looks like
$sharedFoldersLocations = @{
    "sfdirectory/project2/sub1" = "C:\backup\content";
    "sfdirectory/project2/sub2" = "C:\backup\content";
}

If you have Citrix Files for Windows running, a statement like
Get-ChildItem -Directory "S:\Shared Folders\sfdirectory\project2"
might be the right approach. I think there is also a way to do this using the powershell snap in which would probably be more reliable than using the S: drive mapper. I am not really that good at powershell, I've only used it to get this far



Photo of dan becker

dan becker

  • 2 Posts
  • 0 Reply Likes
That's a good Idea. Maybe loop over an array of top-level SF folders, then grab their 1st level sub-folders to built the hash varible that ultimately gets used by doSync fcn. I'd like to do this all in the ps snap-in since it's running on our file server.

We use Citrix Files for Windows but only on client workstations.