Friday, November 29, 2013

Mobile App talking to SharePoint behind TMG

 
Often when one thinks about app development in a corporate environment, ideas strike around business processes that are already existing, functionalities that have already been implemented yet accessibility is limited. When I say limited I mean the mechanism to access that application/functionality is via for example the intranet/extranet. How about have a mobile app that could read/write from your SharePoint extranet which is sitting behind TMG. This was our scenario so here is how I approached it end-to-end.

 

So what will it be? Native, Web, Hybrid? Well since I had HTML5, jQuery and CSS3 skills, I went for mobile web app approach. There was no business requirements for what platforms this app should target so I had the flexibility to opt for the one I could quickly come up with something working. There are a lot of debates around what type of app development should one target and I think it all boils down to the requirements, reach and timeframe that one has.

 

There are a lot of frameworks that can help speed up your mobile app development. I chose jQuery mobile for this instance as it is cross-platform, supports HTML5 and has good documentation to refer to. Cordova for the add-on to access some basic device functions like accelerometer  via JavaScript. I also used a few apple tags that helped me create a splash screen and app icon for iPhone and iPad. The app was also tested on the Andriod phone as well and worked just as fine.

 

The data that I wanted to get on the device was tucked away in a SharePoint list in the company intranet. This list was used to put in leave requests which had a workflow for approval associated with it. I wanted to get this data via SharePoint web service on to the device and show who was on leave . As long as I was on the corporate network, all good but as soon as I had to get it over the internet, I had some trouble with authentication via TMG.

 

TMG had forms based authentication enabled and hence had a forms based authentication page that comes up as soon as I try to access the intranet site over the internet. In order for the TMG to be able to request credentials from the client web service, TMG should throw as 401 code but it would send a 302 redirection code to the login page which does not help when you want to give users a smooth browsing experience. The approach I took was to update the UserAgent of the request to "Microsoft-WebDAV-MiniRedir/6.0.6002". I couldn't do this at the client side as this property cannot be updated using JavaScript. This had to be done on the server side so I wrote a wrapper web service that the client would call and this service in turn would call the SharePoint web service behind TMG by posing as a WebDAV request. TMG understands that the WebDAV request cannot be redirected to forms based authentication page and hence will fall back to Basic Authentication giving a 401 status code and the wrapper service can then respond with the user credentials.

 

I wanted to automate the deployment and code management as much possible and explored some options and found that Windows Azure had an integration with code repositories like BitBucket for auto publishing. As soon as I checked in the code in the repository, the updates got deployed to the cloud. This model of hosting the app code in the cloud is good when you want to manage updates to the code without the users updating the app. This also gives the advantage that all users using your app are on the same version of the code and there is no need to maintain old version functionality if it is not going to be used going forward.

 

Once the app got a decent response and I got some good feedback about it, I then added the functionality to be able to request of leave. This request on the app would result in creating a list item in the leave request SharePoint list which would then trigger the leave approval process.

 

Here is an architectural diagram that explains the approach more visually.

 


There is a big potential for apps like these in organisations that have a lot of information in their SharePoint environments and want to give their users a way to access/interact with the information remotely via a mobile app.

 

Thursday, May 17, 2012

Managed Metadata and the Ampersand Character

Every tied custom code with Managed Metadata Service and got unexpected results when comparing text against the terms in MMS?

 

Managed Metadata services changes the symbol for ampersand to this () not the default (&) text character that we use while typing in the term, so we need to replace it with string character for the comparison to work.

 

Snapshot of MMS Termset and Terms

 

Code snippet to handle this:

 

if (mmTranslatedTerm.Value.Contains(""))

{

              defaultTerm = mmTranslatedTerm.Value.Replace("", "&");

       }

 

Wednesday, May 16, 2012

Caching SharePoint 2010 List Data Programmatically

Performance is one of the key factors for the success of SharePoint implementations and when there is custom code involved to access SharePoint list data, we need to ensure that the data is cached (if it does not get modified too frequently but is used for reference a lot). One of the ways to do so is to use the PortalSiteMapProvider.

 

The following code snippet will help you develop web parts and page layouts that can take advantage of SharePoint’s caching mechanism. The list is assumed to be in the root web in the below example.

 

namespace NameSpaceName

{

    public class ClassName

    {

        protected Repeater repData;

 

        protected void Page_Load(object sender, EventArgs e)

        {

        

            // Create the query.

            SPQuery curQry = new SPQuery();

            curQry.Query = "<Where><Eq><FieldRef Name='Field_x0020_Name' /><Value Type='Text'>" + somevalue + "</Value></Eq></Where>";

 

            //Call the list items from cache/list – provide the list name here

            SiteMapNodeCollection lItems = getDataFromListCache("List Name", curQry);

 

                // Either Bind it to the repeater control

        repData.DataSource = lItems;

       repData.DataBind();

           

//OR

            //Render the list items in a label

  foreach (PortalListItemSiteMapNode pItem in lItems)

        {

                                //do your thing with the label

        }

 

 

        //Function to get cached SP List data from root web

        private SiteMapNodeCollection getDataFromListCache(string listname, SPQuery filter)

        {

            // Get the Root Web object. This is where the list reside.

            SPWeb rWeb = SPContext.Current.Site.RootWeb;

 

            // Create an instance of PortalSiteMapProvider. Used for Caching SP data.

            PortalSiteMapProvider ps = PortalSiteMapProvider.WebSiteMapProvider;

            PortalWebSiteMapNode pNode = ps.FindSiteMapNode(rWeb.ServerRelativeUrl) as PortalWebSiteMapNode;

 

            // Retrieve the items. If the cache for the list is empty, the cache gets created.

            SiteMapNodeCollection pItems = ps.GetCachedListItemsByQuery(pNode, listname, filter, rWeb);

 

            return pItems;

        }

    }

}

 

Imagine calling the list directly. Sound alright but now imagine every user making a call to a list and the number of users being in four/five figures! It will be expensive so make use of caching techniques where possible to optimise.

 

Copying Custom SharePoint Publishing Pages with web parts to another site

Ever tried to copy a custom publishing page with web parts in it to another web.. well copying is not the hard part but the page just refuses to work and throws up the famous SharePoint error - An unexpected error has occurred.

 

So what went wrong? The publishing page is quite different than copying documents around sites. The page and its web parts have references and pointers hardcoded based on the page URL, plus sometimes the content type is not present in the destination pages library which the source page uses  and hence these things make the destination page unusable.

 

One of the projects that I worked on had a requirement to copy pages created in the English sites to sites in other languages so that the users could then localise the content if required. There was no out of the box way to do so!

 

Solution? Our friend… PowerShell!

 

The script loops through the pages library and its folders recursively

-          gets all the pages (picks up pages only if they are ready for propagation - which means they are published and version no is major.0 and the page is not checked out by any user).

-          checks if the page exists in the destination equivalent folder

-          if it exists it checks if the page has been modified by any user or not

-          if not it checks it out and updates the destination page

-          adds the content type used in the page to the destination library (very important)

-          assigns the content type and page layout to the destination file (again very important)

-          assigns the field values of the source page to the destination page

-          copies all webparts from the source page to the destination page (first it deletes the webparts from the destination page to avoid multiple instances of the webparts)

-          checks it in and approves the destination page

-          if the page does not exist does not it crates the page and folder (if folder does not exist) and does the same steps as above.

 

The below PowerShell script works with a configuration list that stores the source and destination site urls but you could modify and use it differently. I didn’t get the chance to optimise the script but please feel free to consolidate the conditional repetitive bits to make it a smaller script.

 

 

#Copy pages to language variation sites from source (EN) site

 

if ( (Get-PSSnapin -Name microsoft.sharepoint.powershell -ErrorAction SilentlyContinue) -eq $null )

{

    Add-PSSnapin "microsoft.sharepoint.powershell"

}

 

#-----------------------------------------------------------------------------------------------------------------------------------------------------------------------#

#add web parts from source page to destination page

function CopyWebPartsP2P($strsUrl, $strdUrl, $dWeb)

{

$spsWpManager = $sourceWeb.GetLimitedWebPartManager($strsUrl, [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)

$spdWpManager = $dWeb.GetLimitedWebPartManager($strdUrl, [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)

 

foreach($spdwebpart in $spdWpManager.Webparts)

      {

           try{

                                $spdWpManager.DeleteWebPart($spdwebpart)

               

            }

            catch

            {

             Write-Host -ForegroundColor Red "An error occurred. Unable to delete webpart " $spdwebpart.Title

            }

      }

 

foreach($spwebpart in $spsWpManager.Webparts)

      {

            Write-Host "web part on " $strsUrl " is " $spwebpart.ID "," $spwebpart.Title ":Zone " $spwebpart.ZoneID ": PartOrder " $spwebpart.PartOrder

         try{

                  $spdWpManager.AddWebPart($spwebpart, $spwebpart.ZoneID, $spwebpart.PartOrder)

            }

            catch

            {

                  Write-Host -ForegroundColor Red "An error occurred. Unable to add webpart " $spwebpart.Title

                  #Write-Host $_.Exception.ToString()

            }

      }

}

#-----------------------------------------------------------------------------------------------------------------------------------------------------------------------#

# Approve the file

function Approve-File($file, $moderated)

{

            Write-Host -ForegroundColor DarkCyan "   Processing file ... Name :" $file.Name

            # Check if the file is checked in

            if($file.Level -eq [Microsoft.SharePoint.SPFileLevel]::Checkout)

            {

                  Write-Host -ForegroundColor DarkGreen "    File is checked out! Checking in ..."

                  $file.CheckIn("checked in by system",[Microsoft.SharePoint.SPCheckInType]::MajorCheckin)

            }

            # If moderation is enabled process the document for approval

            if($moderated)

            {

                  #Approve-Object $file

                  $file.item.ModerationInformation.Status = [Microsoft.SharePoint.SPModerationStatusType]::Approved

                  $file.item.Update()

                                                                      

            }

            Write-Host -ForegroundColor DarkGreen "   Done."

    

}

 

#------------------------------------------------------------------------------------------------------------------------------------------------------------------------#

function IsReadyForPropagation($itm)

{

if($itm.File.MajorVersion -gt 0 -and $itm.File.MinorVersion -eq 0 -and $itm.File.CheckOutType -eq "None")

      {

            return $true

      }

else

      {

            return $false

      }

}

#--------------------------------------------------------------------------------------------------------------------------------------------------------------------------#

function LoopThruFolder($sfldr)

{

            #recursive loop thru source pages library

            $sfldr1 = $cspListItem["Variation Source URL"] + "/" + $sfldr

            if ($cspListItem["Variation Source URL"] -eq "/")

                  {

                        $sfldr1 = $cspListItem["Variation Source URL"] + $sfldr

                  }

            $sfolder = [Microsoft.Sharepoint.SPFolder]$sourceWeb.GetFolder($site + $sfldr1)

        Write-Host "folder = " $sfolder

        $sweb = $sfolder.ParentWeb

            #Write-Host "web = " $sweb

        $slist = $sweb.Lists[$sfolder.ParentListId]

        $squery = New-Object Microsoft.SharePoint.SPQuery

        $squery.Folder = $sfolder

        # Get a collection of items in the specified $folder

       $sitemCollection = $slist.GetItems($squery)

        foreach ($sitem in $sitemCollection)

        {

                  #check if item is file/folder

                  #if folder - recurse

                  if ($sitem.Folder -ne $null)

            {

               Write-Host "Recursive..." $sitem.Folder

               LoopThruFolder $sitem.Folder

            }

            else

            {

                #write-host $sitem.name "*" $sitem.Folder

                        #if file check if current version is major published version

                        # if no - skip

                        if (IsReadyForPropagation $sitem)

                              {

                            

                                    # if yes - check for same page in target site

                                    $destSite = new-object Microsoft.SharePoint.SPSite($site + $cspListItem["Site URL"] + "/" + $LibName)

                                    $destWeb = $destSite.OpenWeb()

                                    $spPubWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($destweb)

                                    $pubpages = $spPubWeb.PagesList

                                    $dLibName = $pubpages.RootFolder.Url

                                    #Write-Host "Dest URL = " $cspListItem["Site URL"]

                                    [Microsoft.SharePoint.SPFile]$dfile = $destWeb.GetFile($sitem.URL.Replace($LibName,$dLibName))

                                    #Write-Host $file.url $file.Exists

                                    if($dfile.Exists)

                                          {

                                          #if is exists

                                          #check if it has been localised. If localised then Modified By will not be SHAREPOINT\system

                                          #check if dest file major version is less than source major version

                                          #so copy only if the file is not already been copied

                                          if ($dfile.ModifiedBy.ToString() -ieq "SHAREPOINT\system" -and $dfile.MajorVersion -lt $sitem.File.MajorVersion)

                                                {

                                                      if ($dfile.CheckOutType -eq "None")

                                                      {

                                                            $dfile.Checkout("Long Term",$null);

                                                      }

                                                      #not localalised yet...hence overwrite the file (else skip)

                                                      #copy file to target pages lib

                                                      write-host "copy " $sitem.URL " to " $sitem.URL.Replace($LibName,$dLibName)

                                                      $dsfldr = $destWeb.GetFolder($sitem.URL.Replace($LibName,$dLibName))

                                                      $spFileCollection = $dsfldr.Files

                                                      $sbytes = $sitem.File.OpenBinary()

$dfile.Delete()

                                                      [Microsoft.SharePoint.SPFile]$fl = $spFileCollection.Add($sitem.URL.Replace($LibName,$dLibName), $sbytes, $true)

                                                      Write-Host "Update content type to " $sitem.ContentType.Name

                                                      $fl.Properties["PublishingPageLayout"] = $sitem.File.Properties["PublishingPageLayout"]

                                                      #[Microsoft.SharePoint.SPContentTypeId]$contenttypeid = $destWeb.ContentTypes[$sitem.ContentType.Name];

                                                                                                                                                                                                                  [Microsoft.SharePoint.SPContentTypeId]$contenttypeid = $sourceweb.Site.RootWeb.ContentTypes[$sitem.ContentType.Name].Id;

                                                     

                                                                                                                                                                                                                if($pubpages.ContentTypes[$sitem.ContentType.Name] -eq $null)

                                                                                                                                                                                                                                                                {

                                                                                                                                                                                                                                                                Write-Host "content type " $sitem.ContentType.Name " does not exists in " $pubpages.Name " hence adding now"

                                                                                                                                                                                                                                                                $pubpages.ContentTypes.Add($sourceweb.Site.RootWeb.ContentTypes[$sitem.ContentType.Name])

                                                                                                                                                                                                                                                                }

                                                                                                                                                                                                                                                               

                                                                                                                                                                                                                if ($fl.CheckOutType -eq "None")

                                                                                                                                                                                                                {

                                                                                                                                                                                                                   Write-Host "Check out..."

                                                                                                                                                                                                                    $fl.CheckOut()

                                                                                                                                                                                                                                                                }

                                                      $fl.Item["ContentTypeId"] = $contenttypeid.ToString()

                                                      $fl.Item.Update

                                                      $fl.Update

                                                     

                                                                                                                                                                                                                  foreach($fld in $sitem.ParentList.Fields)

                                                            {

                                                                  if(!$fld.ReadOnlyField)

                                                                        {

                                                                              Write-Host "field..." $fld.Title

                                                                              $fl.Item[$fld.id] = $sitem[$fld.id]

                                                                        }

                                                            }

                                                                                                                                                                                                                  $fl.Item.Update

                                                      $fl.Update

                                                                                                                                                                                                                  CopyWebPartsP2P $sitem.File.ServerRelativeUrl.ToString() $fl.ServerRelativeUrl.ToString() $destWeb

                                                      Approve-File $fl $pubpages.EnableModeration

                                                      #$destWeb.Update()

                                                      Write-Host "file overwritten to " $dsfldr.Url

                                                }

                                          }

                                    else

                                          {

                                          #if file does not exist

                                          $arrFldrs = $sitem.URL.Replace($LibName,$dLibName).Split("/")

                                          #Write-Host "check folder existance " $sitem.URL.Replace($LibName,$dLibName)

                                          $dsfurl = ""

                                          for($ai=0;$ai -le $arrFldrs.length-1;$ai++)

                                                {

                                                      if ($dsfurl -eq "")

                                                            {

                                                                  $dsfurl  = $dsfurl + $arrFldrs[$ai]  

                                                            }

                                                      else

                                                            {

                                                                  $dsfurl  = $dsfurl + "/" + $arrFldrs[$ai]

                                                            }

                                                      write-host "so far url = " $dsfurl

                                                      if($arrFldrs.length -gt 2 -and ($ai -gt 0 -and $ai -ne $arrFldrs.length-1 ))

                                                            {

                                                            #url has sub folders

                                                            #if folder does not exist

                                                            Write-Host "subfolder = " $arrFldrs[$ai]

                                                            $dsfldr = $destWeb.GetFile($dsfurl)

                                                            #write-host $dsfldr.name "-" $dsfldr.url "-" $dsfldr.Exists

                                                            if (-not $dsfldr.Exists)

                                                                  {

                                                                        #Write-Host "subfolder under" $dsfldr.ParentFolder.Url

                                                                        $pfldr = [Microsoft.SharePoint.SPFolder]$pubpages.ParentWeb.GetFolder($dsfldr.ParentFolder.Url)

                                                                        $newfldr = $pfldr.SubFolders.Add($arrFldrs[$ai]);

                                                                        $newfldr.item.ModerationInformation.Status = [Microsoft.SharePoint.SPModerationStatusType]::Approved

                                                                        $newfldr.item.Update()

                                                                        Write-Host "folder created"

                                                                        $dsfldr = $destWeb.GetFile($dsfurl)

                                                                  }

                                                           

                                                            }

                                                      else

                                                            {

                                                                  Write-Host "Lib/Page=" $arrFldrs[$ai]

                                                                  if($ai -eq $arrFldrs.length-1)   

                                                                  {

                                                                  #copy file to target pages lib

                                                                  write-host "copy " $sitem.File.ServerRelativeUrl " to " $sitem.File.ServerRelativeUrl.Replace($LibName,$dLibName)

                                                                  $dsfldr = $destWeb.GetFolder($dsfurl)

                                                                  $spFileCollection = $dsfldr.Files

                                                                  $sbytes = $sitem.File.OpenBinary()

                                                                  [Microsoft.SharePoint.SPFile]$dfl = $spFileCollection.Add($sitem.URL.Replace($LibName,$dLibName), $sbytes, $true)

                                                                  #$dFileName = $dsfldr.Files.Add($sitem.Name, $sbytes, $true)

                                                                  Write-Host "update content type to " $sitem.ContentType.Name

                                                                  $dfl.Properties["PublishingPageLayout"] = $sitem.File.Properties["PublishingPageLayout"]

                                                                  [Microsoft.SharePoint.SPContentTypeId]$contenttypeid = $sourceweb.Site.RootWeb.ContentTypes[$sitem.ContentType.Name].Id;

                                                                 

                                                                                                                                                                                                                                                if($pubpages.ContentTypes[$sitem.ContentType.Name] -eq $null)

                                                                                                                                                                                                                                                                {

                                                                                                                                                                                                                                                                Write-Host "content type " $sitem.ContentType.Name " does not exists in " $pubpages.Name " hence adding now"

                                                                                                                                                                                                                                                                $pubpages.ContentTypes.Add($sourceweb.Site.RootWeb.ContentTypes[$sitem.ContentType.Name])

                                                                                                                                                                                                                                                                }

                                                                                                                                                                                                                                                if ($dfl.CheckOutType -eq "None")

                                                                                                                                                                                                                {

                                                                                                                                                                                                                   Write-Host "check out..."

                                                                                                                                                                                                                    $dfl.CheckOut()

                                                                                                                                                                                                                                                                }

                                                                

                                                                  $dfl.Item["ContentTypeId"] = $contenttypeid.ToString()

                                                                  $dfl.Item.Update

                                                                  $dfl.Update

                                                                 

                                                                                                                                                                                                                                                                  foreach($fld in $sitem.ParentList.Fields)

                                                                                            {

                                                                                                  if(!$fld.ReadOnlyField)

                                                                                                        {

                                                                                                              Write-Host "field...." $fld.Title

                                                                                                              $dfl.Item[$fld.id] = $sitem[$fld.id]

                                                                                                        }

                                                                                            }

                                                                                                                                                                                                                                                                               

                                                                                                                                                                                                                                                                   $dfl.Item.Update

                                                                  $dfl.Update

                                                                                                                                                                                                                                                                  CopyWebPartsP2P $sitem.File.ServerRelativeUrl.ToString() $dfl.ServerRelativeUrl.ToString() $destWeb

                                                                  Approve-File $dfl $pubpages.EnableModeration

                                                                  #$destWeb.Update()

                                                                  Write-Host "file copied to " $dsfldr.Url

                                                                  }

                                                            }

                                                }

                                          }

                                    $destWeb.Dispose()

                                    $destSite.Dispose()

                              }

            }

        }

}

#parameters to modify

$site = "http://your_site_collection_url"

$LibName = "Pages"

$cListName = "Config List"

#code execution starts here

$spsite = Get-SPSite -identity $site

#Read from site config list to get source and target urls

    $spWeb = $spSite.RootWeb

    $cspList = $spWeb.Lists[$cListName]

    $cspitems = $csplist.items | where-object {($_["Variation Source URL"] -ne $null -and $_["Variation Source URL"] -eq '/')}

    foreach($citem in $cspitems)

      {

    # Casting avoids "Object reference not set to an instance of an object."

    [Microsoft.SharePoint.SPListItem]$cspListItem = $citem

    Write-Host "Dest=" $cspListItem["Site URL"] "; Source=" $cspListItem["Variation Source URL"]

    #for each source-target pair

    #get pages library - append pages to the urls

      if ($cspListItem["Variation Source URL"] -ne "/")

            {

                  $sourceSite = new-object Microsoft.SharePoint.SPSite($site + $cspListItem["Variation Source URL"] + "/" + $LibName)

            }

      else

            {

            $sourceSite = new-object Microsoft.SharePoint.SPSite($site + $cspListItem["Variation Source URL"] + $LibName)

            }

      $sourceWeb = $sourceSite.OpenWeb()

      LoopThruFolder $LibName

      $sourceWeb.Dispose()

      $sourceSite.Dispose()

      }    

 

 

Hope this helps you as much as it did me for my project!