Author: JuliusPIV

Christian, human, consumer of most things IT.

Props to a post I happened to see on reddit around the time I was ready to publish this: https://www.reddit.com/r/Windows10/comments/9d7uqt/yo/

One Method of Simplifying DMP File Analysis

Some time ago we implemented a new VMware Horizon View based VDI  environment and for a while everything seemed to be going well.  Over time customers would periodically call to report that their VDI’s appeared to reboot and in reviewing the machines we noticed some unusual patterns.  After digging around a bit we found cause to be concerned because a significant number of VDI’s were BSOD’ing regularly.

Investigation

Every time a customer reported that their VDI seemed to spontaneously restart I’d check the Event Logs to confirm the restart was due to a bugcheck and not due to Windows Updates.  Once confirmed I’d check for memory dumps then analyze them.  After a while more customers became vocal about the stability of their VDI and the process I was following became tedious:

  1. Check Event Viewer to validate bugcheck vs other process initiated restarts
  2. Check C:\Windows for a MEMORY.dmp
  3. Check C:\Windows\minidump for *.dmp’s
  4. Move the .dmp files to a staging area
  5. Analyze each file with WinDBG

Too many clicks and keyboard action if you ask me. (^_^)

DMP File Analysis Simplification

After evaluating a number of VDI’s I found that in nearly every case the VDI did indeed experience a bugcheck-initiated restart so I stopped performing step 1.  And truthfully, a machine shouldn’t have any .DMP files so even if one was present I wanted to know about it.  With that out of the way I decided to automate the rest of the process in a manner that was good enough for me via a PowerShell script.

Check for & Pull .DMP File(s)

The first function in the script would check key areas for .DMP files, move them to a storage location then populate an array with details on whether or not a .DMP was found.


# centralized bsod collection location
[string]$StorageLocation = '\\Server01\Playground$\CrashDumpStorage'

# store the details of the crash files found
[System.Collections.ArrayList]$CrashDumpDetails = @()

# name of the machine that BSODd<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>
[string]$BSODComputer = 'BSODSystem01'

# path to remote machine
[string]$RemotePath = '\\' + $BSODComputer + '\C$'

# path to data storage location
[string]$MoveDestination = Join-Path $StorageLocation $BSODComputer

# check for memory.dmp
[bool]$MemoryDMP = $false
$MemoryDMPDate = ''
if(Test-Path -Path "$RemotePath\Windows\Memory.dmp" -PathType Leaf -ErrorAction Stop)
    {
        if(!(Test-Path -Path $MoveDestination -PathType Container -ErrorAction Stop))
            {
                New-Item -Path $MoveDestination -ItemType directory -ErrorAction Stop | Out-Null
            }

        [bool]$MemoryDMP = $true

        $MemoryDMPDate = (Get-ChildItem -Path "$RemotePath\Windows\Memory.dmp").LastWriteTime
        Move-Item -Path "$RemotePath\Windows\Memory.dmp" -Destination $MoveDestination -Verbose -Force -ErrorAction Stop
    }

# check for mini dumps
[bool]$MiniDMP = $false
$MiniDMPCount = 0
if(Test-Path -Path "$RemotePath\Windows\Minidump" -PathType Container)
    {
        $MiniDMPCount = (Get-ChildItem -Path "$RemotePath\Windows\Minidump" -ErrorAction Stop | Measure-Object).Count
        if($MiniDMPCount -gt 0)
            {
                if(!(Test-Path -Path $MoveDestination -PathType Container -ErrorAction Stop))
                    {
                        New-Item -Path $MoveDestination -ItemType directory -ErrorAction Stop | Out-Null
                    }

                [bool]$MiniDMP = $true
                Get-ChildItem -Path "$RemotePath\Windows\Minidump" | Sort-Object -Property LastWriteTime | Select -First 5 | % { Move-Item -Path $_.FullName -Destination $MoveDestination -Verbose -Force -ErrorAction Stop }
            }
    }

# add details to arraylist
$CrashDumpDetails += [pscustomobject]@{
    ComputerName = $BSODComputer;
    DumpFileDir = $MoveDestination;
    MemoryDMP = $MemoryDMP;
    MemoryDMPDate = $MemoryDMPDate;
    MiniDMP = $MiniDMP;
    MiniDMPCount = $MiniDMPCount;
}

I would store the output of the function in a variable so the operator (typically me) can see the stats:


$GetCrashDumpFiles | Format-Table -AutoSize

ComputerName    Status DumpFileDir                            MemoryDMP MemoryDMPDate        MiniDMP MiniDMPCount OtherDMP OtherDMPCount
------------    ------ -----------                            --------- -------------        ------- ------------ -------- -------------
BSODMACHINE001  OK     \\Server01\Playground$\BSODMACHINE001      False                        False            0    False             0
BSODMACHINE002  OK     \\Server01\Playground$\BSODMACHINE002       True 9/4/2018 10:35:02 AM    True            2    False             0

Analyzing the .DMP Files

Now that all the .DMP files are in one place I can analyze them with cdb.exe from the Debugging Tools for Windows.

I settled on using an Run Script File instead of hard-coding the command the function allowing me to point to a specific file depending on the operation.  I had a few ‘analysis command files’ (aka run script files) with various commands within, but my core file contains only !analyze -v which is sufficient for my day-to-day tasks.

From there it was just a matter of looping through the directories for .DMP files, running cdb against them and storing the output in a text file for review.


# hold the cdb analysis results
[System.Collections.ArrayList]$AnalysisDetails = @()

# run script file / analysis command file
[string]$AnalysisCommandsFile = "\\Server01\Playground$\DebuggingTools\AutomaticAnalysis\Core.Analysis.1.txt"

# path to cdb.exe
[string]$CDBEXE = '\\Server01\Playground$\DebuggingTools\10\x64\cdb.exe'

# loop through each dmp file and analyze it
foreach($DMPFile in $(get-childitem -path '\\Server01\Playground$\BSODMACHINE002' -Filter *.dmp))
    {
        # generate a log for each dmp file named after both the dump fil and the run script file / analysis command file used
        $Log = (Split-Path -Path $DMPFile.FullName -Parent) + "\" + (Split-Path -Path $DMPFile.FullName -Leaf) + '_' + (Split-Path -Path $AnalysisCommandsFile -Leaf).ToString().Replace('.txt','.log')

        # run cdb against the .dmp fiile using the run script file / analysis command file
        $ServiceTiming = Measure-Command { &$CDBEXE -z "$($DMPFile.FullName)" -c "`$`$<$AnalysisCommandsFile;Q" | Tee-Object -FilePath $Log }

        # capture the exit code
        $CDBExitCode = $LASTEXITCODE

        # populate the analysis results into the array
        $AnalysisDetails += [pscustomobject]@{
            DumpFile = $DMPFile.FullName;
            Date = $DMPFile.LastWriteTime;
            CDBStatus = $CDBExitCode;
            AnalysisDuration = $ServiceTiming;
        }

        Remove-Variable Log,ServiceTiming,CDBExitCode -ErrorAction SilentlyContinue
    }

And you'll see a log file for each .DMP file in the directory:

BSODAnalysis-001

The output again provides some potentially useful stats:


$AnalysisDetails | Sort -Descending -Property Date | Format-Table -AutoSize

DumpFile                                                  Date                 CDBStatus AnalysisDuration
--------                                                  ----                 --------- ----------------
\\Server01\Playground$\BSODMACHINE002\090418-15656-01.dmp 9/4/2018 10:35:25 AM         0 00:00:41.0912218
\\Server01\Playground$\BSODMACHINE002\Memory.dmp          9/4/2018 10:35:02 AM         0 00:00:11.0954740
\\Server01\Playground$\BSODMACHINE002\070218-10000-01.dmp 7/2/2018 9:49:21 AM          0 00:00:29.8022249

And although this is pretty helpful as is, you’re left to open each .log file to see what’s going on.  But that’s no fun.

[Try to] Determine Probable Cause

The bugcheck analysis typically lists the file that was likely the cause of the BSOD right at the top of the log and since the formatting is consistent, we can just extract that from the file.

foreach($LogFile in $(Get-ChildItem -Path '\\Server01\Playground$\BSODMACHINE002' -Filter *.log))
    {
        $ProbablyCausedBy = (Select-String -Path $LogFile.FullName -Pattern "Probably caused by").Line.ToString().Replace('Probably caused by : ','')
        Write-Host "$($LogFile.FullName) BSOD probably caused by: $ProbablyCausedBy"
        Remove-Variable ProbablyCausedBy -ErrorAction SilentlyContinue
    }

That output looks like this:


\\Server01\Playground$\BSODMACHINE002\070218-10000-01.dmp_Core.Analysis.1.log BSOD probably caused by: vm3dmp.sys ( vm3dmp+25900 )
\\Server01\Playground$\BSODMACHINE002\090418-15656-01.dmp_Core.Analysis.1.log BSOD probably caused by: vm3dmp.sys ( vm3dmp+25760 )
\\Server01\Playground$\BSODMACHINE002\Memory.dmp_Core.Analysis.1.log BSOD probably caused by: vm3dmp.sys ( vm3dmp+25760 )

Putting it All Together

With this done, I can now throw an array of machines to kick off the whole process, walk away and come back to some meaningful information.

After much analysis, web scouring, testing and opening a case with VMware we confirmed the root cause of our VDI’s BSOD’ing was due to a flaw [bug] in our version of VMware Tools, 10.1.15.  Basically when someone reconnected to their session, if the display parameters had changed (e.g.: Started the VDI session on their home machine with a resolution of 1024×768 then came into the office, resumed their session with a resolution of 1920×1080; OR Possibly caused by a difference in DPI) it would cause the machine to BSOD.  The customer would never see a BSOD because of the nature of the VDI client – it would just look like a [spontaneous] restart.

The fix was to upgrade to VMware Tools 10.2.x.   But VDA 7.2.x wasn’t compatible with VMware Tools 10.2.x which meant upgrading the VDA to 7.4.x.
The correct remediation procedure looks like this:

  • Uninstall VDA 7.2.x
  • Uninstall VMware Tools 10.1.x
  • Reboot
  • Install VMware 10.2.x
  • Reboot
  • Install VDA 7.4.x
  • Reboot

If you’re using AppVolumes or a GPU then you’ll need to modify this accordingly because order of operations is key!

Because this process required multiple restarts, and more importantly killed one’s ability to reconnect to the machine, I created a Task Sequence to handle this.  Once the VDI’s were updated, they were rock solid.

The full .DMP analysis PowerShell script can be found here.

In Conclusion

Although this script may not be perfect, it meets my needs of simplifying what would normally be a manual process, allowing me to focus more on providing great timely customer service and just ‘getting it done‘ as efficiently as possible.  In fact, I’ve used this script to analyze .DMPs on physical assets making that process easy.  In a future version of this I’d like to intelligently handle processed .DMP files and logs so we’re not constantly analyzing the same set of files. (i.e.: After analysis move them into a ‘Processed’ directory or something)

I may have been able to simplify this a bit by using existing products, like Nir Sofir’s BlueScreen View or something else I’m not yet aware of.  But there’s often a challenge in introducing new software into an environment and many NirSoft utilities are classified as PUA’s so they’re killed with extreme prejudice.  So, I try to stick with the tools Microsoft provides to get the job done when it makes sense to do so.

Remember, this is not the way, it’s just a way.

Good Providence to you!

Advertisements

Options for Nerfing Software Center

Note: I started this post several months ago but held off on posting it.  Since then, the circumstances have changed virtually eliminating the need for accessing Software Center not nearly as important as it once was.  That said, this may be a non-issue for you but I figured I’d at least share what I discovered.

In an Office Several Months Ago…

We stood up a new Citrix XenApp environment leveraging SCCM to build the Session Hosts over Machine Creation Services (MCS).  As such, Software Center was present posing a potential problem in trying to keep a pristine Citrix environment: We certainly can’t have our customers installing software willy-nilly now!  We had some discussions internally about this and there were two key objectives:

  1. We need to prevent our customers from installing software be it from Software Center or the Application Catalog
  2. We need to allow access to Software Center in order for our server admins to kick off Windows Updates, and install applications as necessary, from Software Center.

To address this, I started exploring what options were available to avoid unnecessarily drawing attention to the Software Center and allowing admins access for maintenance purposes.

The Software Center Shortcut

The first course of action would be to delete the ‘Microsoft System Center’ directory in:

%ALLUSERSPROFILE%\Microsoft\Windows\Start Menu\Programs\

Takeaway

Truthfully, deleting the shortcut would immediately obscure Software Center from search which should prevent most customers from launching it but it’s not a silver bullet.

The softwarecenter: URI/protocol

If someone is a command line warrior they may know about launching Software Center via the softwarecenter: uniform resource identifier (URI).  As you probably already know, this URI is super handy for sending shortcuts to customers to specific applications via email or an Intranet page but it’s presence naturally poses a problem because leaving it available, provides a valid Software Center entry point.

In order to eliminate this shortcut, you need the protocol details from the registry:

Delete the ‘softwarecenter’ key from HKEY_CLASSES_ROOT:

reg delete "HKEY_CLASSES_ROOT\softwarecenter" /f

Delete the ‘softwarecenter’ key from HKEY_LOCAL_MACHINE\Software\Classes:

reg delete "HKEY_LOCAL_MACHINE\SOFTWARE\Classes\softwarecenter" /f

Takeaway

This would help to address those cases where we send out an email communication with a link to some software we want to highlight and the reader is in Citrix.
This also breaks it for those command line warriors that know about softwarecenter:.

Software Center EXE’s

Even with all the work done above, if a notification appears and a customer clicks the notification, it would still launch Software Center.  Also, if someone happened to know the path to Software Center they could just launch it manually.  The former should be handled at a higher level (more on that below) but the latter can technically be addressed on the host itself but remember:

Just because we can do something doesn’t mean we should.

Although it is not likely that our customers know about this, there are two executables for Software Center:

  • %WinDir%\CCM\SCClient.exe
  • %WinDir%\CCM\ClientUX\SCClient.exe

The one that’s presented to customers depends on the version of Software Center you’re using:

  1. If you’re using the older Software Center UI the former is launched.
  2. If you’re presenting the newer Software Center UI, the latter is launched.

You can figure out which one is the default by querying this registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Classes\softwarecenter\shell\open\command

To prevent customers from running them we have some options:

  1. Delete the EXE’s
  2. Rename the EXE’s
  3. Apply permissions on the EXE’s to prevent non-Administrators from executing
  4. Apply permissions on the parent directories to block access to non-Administrators

Takeaway

In my humble opinion, this is tricky for a variety of reasons:

  1. Assuming you did one of the above, if a customer tried to access Software Center, perhaps via notification or otherwise, some unfriendly errors would be thrown making for a poor customer experience.
    .
  2. The method you choose depends on which version you’re using: old or new.
    Since I’m a believer of making things ‘ID 10 T’ proof, I would use the same approach for both to eliminate any potential gaps.
    .
  3. Depending on the approach you take, there may be some potential ramifications.
    I don’t believe deleting or renaming is wise but it is an option.
    .
  4. When the ConfigMgr client performs it’s health checks, would it undo all of this?
    .
  5. If someone repairs the client, would that undo all of this?
    .
  6. When the client is upgraded, would that undo all of this?
    .
  7. No idea on the level of supportability of these actions.

For me there are just too many unknowns for this to be a viable solution.

Fixing it at a Higher Level

This in my opinion should be the real starting point as it’s one half of the ‘meat & potatoes’ of this post.

Client Settings Level

Create a new Client Settings policy deployed to the Device Collection you created above with one setting change within the Computer Agent Setting:

Install Permissions

Set ‘Install permissions’ to something other than ‘All Users’ which is the default:

  1. Only Administrators: Users must be a member of the local Administrators group.
  2. Only Administrators and primary users: Users must be a member of the local Administrators group, or a primary user of the computer.
  3. No Users: No users signed in to a client computer can initiate the installation of Software from the Application Catalog or Software Center and Software Updates or Task Sequences from the Software Center.

Takeaway

By setting it to ‘Only Administrators’ you’ve effectively killed Software Center’s usefulness for the every-day Citrix user, even if they tried to initiate the installation it from the Application Catalog.  That is of course assuming your Citrix users are not local administrators on Citrix servers.  (Hint: they shouldn’t be!)

OPTIONAL: Show Notifications

Set ‘Show notifications for new deployments’ to ‘No’ versus the default of ‘Yes’:

Choose Yes to display a notification for deployments available for less than a week. This message appears each time the client agent starts.

Takeaway

This will help prevent unnecessary notifications that would draw unnecessary attention to Software Center.

Collection Level

Put all these special machines into a Device Collection and only deploy what you need to deploy to those machines.

Takeaway

This is probably just leading practice advice but you should keep your deployments tight so they’re reaching the intended audience.  There isn’t much to gain in making Office 365 available to ‘All Systems’ if it would only ever be installed on Windows 10 Workstations and not on machines running Windows 7 or a Windows Server OS.

Deployment Level

When deploying things to the special Device Collection you created above, make sure you hide the notifications and optionally hide them from Software Center altogether.

Takeaway

This will help prevent unnecessary notifications that would draw unnecessary attention to Software Center.

Application Level

On the Deployment Type, add an Operating System requirement so that it must be non-Server OS.

Alternatively you can create your own Create a Global Condition specifically for these machines and add it to your applications.

Takeaway

This is a double edged sword, a bit tedious and potentially overkill considering the alternative options that are easier to implement.

Excluding Software Center During Installation of ConfigMgr Client

This in my opinion is the other half of the ‘meat & potatoes’ of this post.

If you’re installing the ConfigMgr Client manually you can exclude Software Center at installation time by running the following command line:

ccmsetup.exe /excludefeatures:clientui

But since all of these machines were built via an SCCM OSD Task Sequence we’re relying on the stock ‘Setup Windows and ConfigMgr’ Task Sequence step to get the ConfigMgr client loaded.  Since that’s all handled internally, we don’t have a way to alter the command line to add that specific switch.

Although to be fair, we can technically run this during or after OSD completes, I’m just not a big fan of doing things like that:

ccmsetup.exe /forceinstall /excludefeatures:clientui

But that was the golden nugget I needed!  I figured since there’s an argument for this to remove it, maybe there’s something we could do at installation time to prevent it from running.  A little sleuthing (i.e.: enabling verbose Windows Installer logging, running the above commands then reviewing the log files generated) revealed two new – to me anyway – MSI properties that appeared to be relevant:

CCMEXCLUDEDFEATURES=ClientUI REMOVE=ClientUI

Bingo was his name!  At the very least the CCMEXCLUDEDFEATURES=ClientUI property/value pair can be added to the ‘Setup Windows and ConfigMgr’ Task Sequence step so we can prevent Software Center from coming down in one fell swoop.

After re-imaging a server:

  • the Software Center icon was not present
  • the softwarecenter: URI/protocol was not functional
  • the Software Center Client executables were not present

Takeaway

This is arguably the best method for eliminating all avenues to Software Center since it’s completely supported and will persist through client upgrades, health checks and all that.

BONUS: Software Center Portability

Now that we’ve sorted out the customer facing pieces we needed to shift our focus to our server admins.

Early on in the Citrix project there was a need to make sure a Citrix/server admin could access Software Center to kick off the installation of patches or install new or updated core software the Citrix server needed.  As a result of the change made above we killed their ability to do that.  Fortunately Software Center appears to be ‘portable’ in that we can copy up the %WinDir%\CCM\ClientUX directory to a network share and run it from there and it works as expected.

There are however two concerns with this approach::

  1. Supportability – If Software Center doesn’t work properly, we may not be able to get support from Microsoft on this.
  2. Someone will have to maintain this repository each time we upgrade ConfigMgr or apply a ConfigMgr patch.

We’ve been using this configuration for a while now and so far, it has been working fine without issue.  Good news is we’re now relying on automation to get these server’s patched so there isn’t as much reliance on Software Center as there once was.

In Closing

Is this the end all be all?  No, I doubt it.  Clearly there are many ways to ‘nerf’ or limit access to Software Center depending on the nature of the situation and hopefully one of these helps you in that endeavor.  If you are using another method, tell me about it in the comments!

For now I bid you Good Providence!

The ol' 'Quick & Dirty' method - Image provided by Max Pixel

MDT Performance Boost Part 2

I need all the help I can get in terms of speeding up the build & capture process so in addition to the steps mentioned in my previous post, I use this “one weird trick” (smh) to help speed up the actual WIM capturing or generation process.

Speedup Process Overview

  • Copy the base install.wim of your B&C OS to your capture location
  • Rename the .WIM to something meaningful like 1809_install.wim
  • Use that same name for the WIM file name for the capture
  • During B&C your image will be added to the existing WIM
  • Split the captured image into it’s own .WIM
  • Delete the index

Baseline Testing (Optional)

This isn’t required but it’s the only way to validate that the speedup procedure actually works.   In order to do that, you’re just going to start by performing a regular Build & Capture using the normal process to see how long the actual .WIM capture process takes.  This will be our baseline number.  In my test lab, after running several B&C’s I’m finding the capture process takes anywhere from 35 to 40 minutes in my lab.

MDTSpeedup2-001

Apply the Speedup Procedure

To keep things simple, let’s say we’re working on capturing an 1809 image.

Copy the stock 1809 install.wim to your Captures directory and rename it to something meaningful like 1809_install.wim.

Check the details of the WIM you just copied:

dism /get-wininfo /image:"\\MDTServer\DeploymentShare$\Captures\1809_install.wim"

Deployment Image Servicing and Management tool
Version: 10.0.14393.0

Details for image : \\MDTServer\DeploymentShare$\Captures\1809_install.wim

Index : 1
Name : Windows 10 Education
Description : Windows 10 Education
Size : 14,356,142,049 bytes

Index : 2
Name : Windows 10 Education N
Description : Windows 10 Education N
Size : 13,548,111,095 bytes

Index : 3
Name : Windows 10 Enterprise
Description : Windows 10 Enterprise
Size : 14,356,212,795 bytes

Index : 4
Name : Windows 10 Enterprise N
Description : Windows 10 Enterprise N
Size : 13,548,004,622 bytes

Index : 5
Name : Windows 10 Pro
Description : Windows 10 Pro
Size : 14,356,028,734 bytes

Index : 6
Name : Windows 10 Pro N
Description : Windows 10 Pro N
Size : 13,547,960,587 bytes

Index : 7
Name : Windows 10 Pro Education
Description : Windows 10 Pro Education
Size : 14,356,071,811 bytes

Index : 8
Name : Windows 10 Pro Education N
Description : Windows 10 Pro Education N
Size : 13,548,039,957 bytes

Index : 9
Name : Windows 10 Pro for Workstations
Description : Windows 10 Pro for Workstations
Size : 14,356,106,696 bytes

Index : 10
Name : Windows 10 Pro N for Workstations
Description : Windows 10 Pro N for Workstations
Size : 13,548,075,292 bytes

Index : 11
Name : Windows 10 Enterprise for Virtual Desktops
Description : Windows 10 Enterprise for Virtual Desktops
Size : 14,356,177,402 bytes

The operation completed successfully.

Now perform a B&C to get the updated timing details. In my lab, I’m seeing it take about 9 minutes*.

MDTSpeedup2-002

Once the .WIM is created, verify the new index was created:


dism /get-wininfo /image:"\\MDTServer\DeploymentShare$\Captures\1809_install.wim"

Deployment Image Servicing and Management tool
Version: 10.0.14393.0

Details for image : \\MDTServer\DeploymentShare$\Captures\1809_install.wim

Index : 1
Name : Windows 10 Education
Description : Windows 10 Education
Size : 14,356,142,049 bytes

Index : 2
Name : Windows 10 Education N
Description : Windows 10 Education N
Size : 13,548,111,095 bytes

Index : 3
Name : Windows 10 Enterprise
Description : Windows 10 Enterprise
Size : 14,356,212,795 bytes

Index : 4
Name : Windows 10 Enterprise N
Description : Windows 10 Enterprise N
Size : 13,548,004,622 bytes

Index : 5
Name : Windows 10 Pro
Description : Windows 10 Pro
Size : 14,356,028,734 bytes

Index : 6
Name : Windows 10 Pro N
Description : Windows 10 Pro N
Size : 13,547,960,587 bytes

Index : 7
Name : Windows 10 Pro Education
Description : Windows 10 Pro Education
Size : 14,356,071,811 bytes

Index : 8
Name : Windows 10 Pro Education N
Description : Windows 10 Pro Education N
Size : 13,548,039,957 bytes

Index : 9
Name : Windows 10 Pro for Workstations
Description : Windows 10 Pro for Workstations
Size : 14,356,106,696 bytes

Index : 10
Name : Windows 10 Pro N for Workstations
Description : Windows 10 Pro N for Workstations
Size : 13,548,075,292 bytes

Index : 11
Name : Windows 10 Enterprise for Virtual Desktops
Description : Windows 10 Enterprise for Virtual Desktops
Size : 14,356,177,402 bytes

Index : 12
Name : W10_1809_Entx64
Description : 
Size : 24,935,992,683 bytes

The operation completed successfully.

Success!

 

*So, What’s the Catch?

Alright – so maybe I cut a corner or two.

I’ll admit, it might not be helpful to have one giant .WIM with 12+ indexes, so you’d probably want to export that newly created index to it’s own .WIM:

dism /export-image /sourceimagefile:\\MDTServer\DeploymentShare$\Captures\1803_install.wim /sourceindex:12 /destinationimagefile:\\MDTServer\DeploymentShare$\Captures\1803_bnc.wim

I find this process takes about 90 seconds putting us at about 10m30s.

And if you’re doing that you might also want to delete the new index you created in the original install.wim:

dism /delete-image /imagefile:\\MDTServer\DeploymentShare$\Captures\1803_install.wim /index:12

This process takes about 2 seconds putting us at about 10m32s.

On the surface, capturing or generating a WIM in 8-10 minutes sounds like some appreciable gains but this is where the corner cutting begins.

Like a fine wine, I find this process gets [somewhat] better with age:

  • Without any funny business my first capture typically takes anywhere from 35 to 40 minutes.
  • When I use this technique I might shave 5-10 minutes off my first capture depending on what’s in there.  It’s something but not significant.
  • However, when I do my second (third, fourth etc.) capture with this technique – like when I’m doing another B&C to include this/last month’s patches and/or updated software etc. – that’s when I begin to appreciate the speed difference and now the actual WIM capture/generation process takes ~8 to ~11 minutes.

After the first big capture – thick image with everything in it – that usually becomes it’s own .WIM and the differential .WIMs are appended to it.  From here we can either export those differential indexes into their own .WIMs or we leave them in the ‘master’ .WIM which allows for easy regression to a previous ‘version’ by just using a lower number index in the Task Sequence.

In Conclusion

I don’t think this is ground breaking and there are some caveats – specifically being that you have go through one capture before you can truly realize the benefits – but like I said, I need all the help I can get so every little bit counts!

So hopefully this “one weird trick” helps you 🙂

Good Providence!

MDT Performance Boost Part 1

I need all the help I can get in terms of speeding up the build or build & capture process so I leverage the available hardware to help speed things up.  But before we get to it, let’s give credit where it’s due!

Recommended Reading

I can’t recall exactly when I first discovered those posts, but I didn’t have robust hardware at the time; In fact, I didn’t have SSD’s but I did have access to servers with significant amounts of RAM which is why this was so attractive.  I do want to warn you however that you should curb your expectations as your mileage may vary.

Speedup Process Overview

  • Use a RAM Disk or SSD
  • Give your B&C VM plenty of resources

It’s all About the Hardware

I’m blessed in that I do a lot of my development work on fairly robust hardware:

  • At home: Dell PowerEdge R710 with Dual E5649’s, 192GB of RAM and 8x146GB 15K SAS Drives in a RAID 10 configuration
  • On the go: Lenovo W541 i7-4810MQ with 32GB of RAM, 2x Sandisk Extreme Pro SSD’s in RAID 1 for dev work (VM’s, MDT etc.)
  • At work: Lenovo P500 E5-1620 v3, 128GB of RAM, 2x SSD’s in RAID 1 for dev work (VM’s, MDT etc.)

All of this allows me to leverage the additional cores and RAM to squeeze a bit of extra performance in build & capture scenarios.

Hyper-V Host Configuration

Because my host has gobs of RAM, I create a RAM Disk (or RAM Drive if you prefer) that is ultimately used to store ‘build & capture’ virtual machine’s .VHDX.  I typically allocate 60GB becuase I’m testing full load out scenarios but you can get away with considerably less for more bare bones builds.  There are a variety of RAM Disk solutions out there including, but not limited to:

  • ATI Radeon RAMDisk / DataRAM
  • ImDisk
  • MSI RAMDisk
  • SoftPerfect RAM Disk
  • StarWind RAM Disk

I landed on the no-frills StarWind RAM Disk product because it’s been maintained over the years and at the time of writing – as well as for the past several years – StarWind’s RAM Disk product has made available free of charge, doesn’t have any limitations I’m aware of and most importantly it just works.  Oh – and by “free of charge” I mean yes it doesn’t cost any money but they do ask that you offer up your personal details (name, email address etc.) “to accommodate your requests, keep in touch with you, and offer a better customer experience.

I know I know, this may be a pain point for you but this is a requirement as the download link comes via email.  I can say that I personally can’t remember the last time StarWind emailed me so I feel comfortable offering it up.  If you don’t like sacrificing personal details, consider the other products on the market most of which offer additional features like being able to save and restore the state of your RAM disk.

I do want to point out that this only works if you have excess RAM to spare.  If you don’t have enough RAM to create a RAM Disk for VM use, the next best thing would be to add an SSD to your machine or swap your mechanical drive for an SSD.  In a pinch I suppose you could use a large (64GB+) microSD/SD card or USB 3.0 thumb drive, but I don’t recommend that as a long term solution.  Just because you can doesn’t mean you should though! (^_^)

Virtual Machine Configuration

When creating a ‘build & capture’ VM, I typically give it 4 virtual processors and 8GB of RAM.  I’ve done B&C’s with 1 & 2 virtual processors an 2-4GB of RAM and the performance isn’t great.  You might be asking yourself:

Why not use 8 virtual processors and 16GB of RAM?

You absolutely can, but there is a point where the performance increase levels off at which point the resource cost outweighs the benefit. (i.e.: you’re not making good use of your resources.)  This 4 virtual CPU and 8GB of ram has worked well for me so far which is why I haven’t bothers measuring the performance benefits of 4+ virtual processors and 8+ GB of RAM.  If you’re feeling up to the task, I’ll leave you to do that and you might as well play around with the scheduler to ensure that B&C machine has the highest priority.

Putting It All Together

Installing StarWind RAM Disk

Download & install StarWind RAM Disk – it’s very much double click the EXE and Next, Next, Next to completion.

Configuring StarWind RAM Disk

Launch RAM Disk

StarWind_RAMDisk001

Click ‘Add Device’ to create a new RAM Disk and configure it to your liking

StarWind_RAMDisk002

The process may appear to hang while it creates the RAM Disk.  Go refill you cup of water or visit the loo and it’ll be done by the time you get back.

StarWind_RAMDisk003

When it’s complete you’ll see the new device and you should receive a notification

StarWind_RAMDisk004StarWind_RAMDisk005

Configuring the RAM Disk

After creation the new RAM disk will be offline so fire up Disk Management (or Server Manager whatever) to finish configuring it:

  1. Online the disk
  2. Initialize the disk
  3. Select GPT (or MBR if you prefer)
  4. Create a new simple volume
  5. Assign it a letter – I like to use R: for RAM Disk
  6. Use the NTFS file system
  7. OPTIONAL: Set the volume label – I like to use RAMDisk
  8. Select quick format

StarWind_RAMDisk006

Configuring Your B&C VM

Now that the disk is ready for use, it’s time to configure your B&C VM

If you don’t already have a B&C VM

  1. Create a new BnC VM
  2. Give it at least 8GB of RAM
  3. When asked where to place the .VHDX, change the path to the RAM disk
    • StarWind_RAMDisk007
  4. Once finished, edit the VM settings and give it at least 4 virtual processors

If you already have a BnC VM

  1. Edit the VM settings
  2. Give it at least 2 virtual processors; 4 if you can spare it
  3. Give it at least 4GB of RAM; 8 if you can spare it
  4. When asked where to place the .VHDX, change the path to the RAM disk
    • StarWind_RAMDisk008
    • StarWind_RAMDisk009
    • StarWind_RAMDisk010

That’s it – now you’re cooking with gas!  Start your VM and test an image.

In Closing

When I initially did this, I did notice an appreciable decrease in machine build out time.  We’re not talking going from 60 minutes to 20 minutes, but it was a noticeable improvement nonetheless.  If I can improve my build times by up to 15-25% then I’ll take it!  Now that we have a mostly automated build process, speeding it up it just gravy allowing for more B&C’s throughout the day.

In any event, I hope you find this tip as useful as I do!

Good Providence to you!

MDT Tutorial Part 12: Wrap Up

Living Table of Contents

What These Guides Are:
A guide to help give you some insight into the MDT imaging & troubleshooting process in general.

What These Guides Are Not:
A guide to create a one-size fits all solution that fixes all the issues you’re ever going to encounter.

Why Bother with My Tutorials?

You totally don’t have to take my word on it!  I’m not world-reknowned like others listed on the Resources page – heck I’m not even county-reknowned so you totally should scrutinize the content and provide me with any constructive criticism you may have. 🙂

However I’ve learned from experience, as well as from others in the circles I travel in, that although turn-key solutions are great, once implemented those who implemented them are not fully equipped to maintain the environment.

Please do not misconstrue what I’m saying here: There is nothing wrong with the turn-key solutions out there!  It’s simply that we’re not all at the same level from a technical perspective and we all have knowledge gaps.  But it’s that combination that makes it challenging for some to support those turn-key solutions.

For me anyway I find that having a good base and some reference points better equips me for the road that lies ahead.  And when something breaks it’s an excellent opportunity to review my depth of knowledge on the subject to troubleshoot my way back into a functioning state.  But that’s me and it may not be you.  Maybe you’re just some savant and it comes naturally.

If you were brave enough to go through this process and successfully built, captured & deployed images, then you should have sufficient functional knowledge to efficiently use the turn-key solution below.

More Labs & Test Environments from Microsoft

Turn-Key Solution from Microsoft: Microsoft 365 Powered Device Lab Kit (Formerly Windows 10 Deployment and Management Lab Kit)

The Microsoft 365 powered device lab kit (formerly Windows 10 Deployment and Management Lab Kit) is a complete pre-configured virtual lab environment including evaluation versions of:

  • Windows 10 Enterprise, version 1803 (Windows 10 April 2018 Update)
  • System Center Configuration Manager, version 1802
  • Windows Assessment and Deployment Kit for Windows 10, version 1803
  • Microsoft Deployment Toolkit
  • Microsoft Application Virtualization (App-V) 5.1
  • Microsoft BitLocker Administration and Monitoring 2.5 SP1
  • Windows Server 2016
  • Microsoft SQL Server 2014
  • Connected trials of:
    • Office 365 Enterprise E5
    • Enterprise Mobility + Security

The best part is that it also includes illustrated step-by-step lab guides to take you through multiple deployment and management scenarios, including:

Servicing

  • Windows Analytics Update Compliance
  • Servicing Windows 10 with Configuration Manager
  • Servicing Office 365 ProPlus with Configuration Manager

Deployment and management

  • Modern Device Deployment
  • Modern Device Management with AutoPilot
  • Modern Device Co-Management
  • Office 365 ProPlus Deployment
  • BIOS to UEFI Conversion
  • Modern Application Management with Intune
  • Enterprise State Roaming
  • Remote Access (VPN)

Security

  • Windows Information Protection
  • Windows Defender Advanced Threat Protection
  • Windows Defender Application Guard
  • Windows Defender Application Control
  • Windows Defender Antivirus
  • Windows Defender Exploit Guard
  • Windows Hello for Business
  • Credential Guard
  • Device Encryption (BitLocker)
  • Remote Access (VPN)

Compatibility

  • Windows App Certification Kit
  • Windows Analytics Upgrade Readiness
  • Browser Compatibility
  • Windows App CertificationKit
  • Desktop Bridges

 

This is an amazing kit because of its holistic approach to helping IT Pros transition to the ‘Modern Desktop’.  As such it’s a hefty download and the hardware requirements are steep.

Turn-Key Solution from Johan Arwidmark: Hydration Kit

Johan has been churning out Hydration Kits since as far back as ConfigMgr 2007 SP2 and MDT 2010 so he’s one of the top 5 go-to’s for this sort of thing.

From the author:

This Kit builds a complete ConfigMgr CB 1702, and ConfigMgr TP 1703, with Windows Server 2016 and SQL Server 2016 SP1 infrastructure, and some (optional) supporting servers. This kit is tested on both Hyper-V and VMware virtual platforms, but should really work on any virtualization platform that can boot from an ISO. The kit offers a complete setup of both a primary site server running ConfigMgr Current Branch v1702 (server CM01, as well as a primary site server running ConfigMgr Technical Preview Branch v1703 (server CM02). You also find guidance on upgrading current branch platform to the latest build.

There’s plenty of documentation in the link:

https://deploymentresearch.com/Research/Post/580/Hydration-Kit-For-Windows-Server-2016-and-ConfigMgr-Current-Technical-Preview-Branch

 

Turn-Key Solution from Mikael Nystrom: Image Factory for Hyper-V

Mikael has been working on this for at least 2 years and is another in the top 5 go-to’s for this sort of thing.

From the author:

The image factory creates reference images using Microsoft Deployment Toolkit and PowerShell. You run the script on the MDT server. The script will connect to a hyper-v host specified in the XML file and build one virtual machine for each task sequences in the REF folder. It will then grab the BIOS serial number from each virtual machine, inject it into customsettings.ini, start the virtual machines, run the build and capture process, turn off the virtual machine and remove them all.

There’s some great documentation here but plenty to browse through in the link:

https://github.com/DeploymentBunny/ImageFactoryV3ForHyper-V

 

Turn-Key Solution from Mike Galvin: Image Factory for Microsoft Deployment Toolkit

I happened upon this by accident but I like how straight forward it appears to be.

From the author:

Recently I’ve been looking into automating my image build process further using PowerShell to create an Image Factory of sorts. There are other similar scripts that I’ve found online, but I like the relatively simple and straightforward method I’ve developed below. If like me you’re looking to automate your image build process so that you can have a fully up-to-date image library after patch Tuesday each month, then this may be for you.

This link is the original post where the script is introduced but I strongly urge you to start with the below link then work your way backwards from there.

https://gal.vin/2017/08/26/image-factory/

 

Turn-Key Solution from Coretech: Image Factory

This unfortunately is not something I can link without burning a bridge.  This solution is only obtainable from [select (?)] Coretech training classes or simply if Kent Agerlund likes you.  It’s conceptually similar to many others here, relying on Hyper-V to spin up VM’s, start them, boot, start a specific build & capture task sequence, run through the task sequence then destroy the VM.  Done.

 

In Closing

As you can see there are plenty of training, lab and turn key imaging solutions out there but to quote Browning:

Image the whole, then execute the parts
          Fancy the fabric
Quite, ere you build, ere steel strike fire from quartz,
          Ere mortar dab brick!

In other words, get the total picture of the process; and I argue there are two ways of doing that: with a telescope and a microscope.

Start with the telescope to look at all of it from thirty-thousand feet and you’ll see almost all these require a dab of elbow grease to get going.  In fact, most make assumptions about the environment as well as a certain amount of proficiency with the various technologies being used: PowerShell, MDT, Hyper-V, SCCM, Active Directory etc.

In the same vain that I’d rather learn to drive a manual transmission on an old beater car before I jump into an STI, GT3 RS, or <insert other absurdly expensive manual transmission vehicle here> , it makes more sense to have a slightly more-than-basic understanding of how the technology works before diving in head first.

Good Providence to you!

An authentication error has occurred. The function requested is not supported. This could be due to CredSSP encryption oracle remediation. CVE-2018-0886

Problem:

I’ve been working furiously on some Citrix XenApp stuff recently on shiny new Server 2016 boxes.  Yesterday was a productive day and all was well.  With it also being Patch Tuesday and my machines part of the Patient Zero Device Collection targeted for updates I received May’s patches last night/this morning.

Today, when I attempted to RDP into Server 2016 boxes I received the following error:

CredSSPOracle

[Window Title]
Remote Desktop Connection

[Content]
An authentication error has occurred.
The function requested is not supported

Remote computer: <remote computer>
This could be due to CredSSP encryption oracle remediation.
For more information, see https://go.microsoft.com/fwlink/?linkid=866660

[OK]

Cause:

This is intentional and I urge you to direct your attention to the URL in the message: https://go.microsoft.com/fwlink/?linkid=866660

Cliff’s Notes version of the cause from the article:

  • The initial March 13, 2018, release updated the CredSSP authentication protocol but did NOT enforce the new version of the CredSSP protocol.
  • The April 17, 2018, Remote Desktop Client (RDP) update in KB 4093120 enhances the error message that is presented when an updated client fails to connect to a server that has not been updated.
  • The May 8, 2018, update makes the new updates CredSSP protocol mandatory.
    This intentional change adjusts the default setting from ‘Vulnerable’ to ‘Mitigated’.

Solution:

In reviewing the interoperability matrix there are only a few blocked scenarios:

  1. Server Patched ‘Force updated clients’ + Clients Unpatched = Blocked
  2. Server Unpatched + Clients Patched ‘Force updated clients’ = Blocked
  3. Server Unpatched + Clients Patched ‘Mitigated’ = Blocked

Well I know my client is patched so that rules out Scenario 1, making it clear our Server 2016 servers are missing KB 4103723.

Solution: Patch your servers!

Fauxlution

This is not a solution.  It’s a fake solution or as I like to call them faux-lutions.

So is there a workaround?  Sure.  So in my particular scenario, I would set the patched client(s) to ‘Vulnerable’  which means that I would then be exposing remote servers to attacks by supporting fallback to insecure versions.

Arguments can be made either way to justify this but I don’t think its wise:

  • It negatively affects our security posture
  • I’m human thus prone to forgetting things and then I’ll never undo it.

I’d rather submit an emergency change request to patch the servers.

In fact, Microsoft’s recommendation is to set AllowEncryptionOracle on clients and server computers as soon as possible to one of the following:

  • Force updated clients = 0
  • Mitigated = 1

But if you want to go down this slippery slope at your own risk, set on your patched client(s), set AllowEncryptionOracle to 2 and you’ll be able to connect to your unpatched server(s):


reg add "HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\CredSSP\Parameters" /v AllowEncryptionOracle /d 2 /t reg_dword

The documentation states a reboot is required but in testing, a reboot is not required.

References:

  1. CVE-2018-0886 | CredSSP Remote Code Execution Vulnerability
  2. Windows 10 1803 May 8, 2018—KB4103721 (OS Build 17134.48)
  3. Windows 10 1709 May 8, 2018—KB4103727 (OS Build 16299.431)
  4. Windows 10 1703 May 8, 2018—KB4103731 (OS Build 15063.1088)
  5. Windows 10 1607 & Server 2016 May 8, 2018—KB4103723 (OS Build 14393.2248)

 

Whatever route you take, I bid you Good Providence!

Configure Network Settings Error: Enter the DNS server IP address.

Problem:

You’re imaging a machine on a network segment not serviced by a DHCP server.

On the first screen of the Task Sequence Wizard – the one that says ‘Welcome to the Task Sequence Wizard‘ – you click the ‘Configure Network Settings‘ button and enter the ‘Configure Static IP Network Settings‘ screen.

After entering everything in correctly and clicking ‘OK‘, you receive the following error:

EnterTheDNSServerIPAddress

Configure Network Settings Error: Enter the DNS server IP address.

Troubleshooting:

  • The network configuration you entered is correct.
  • You reboot, try again, same error.
  • Maybe you try another machine and it works
  • Maybe you try another machine and it doesn’t work.

Potential Cause 1:

In my limited experience, this seems happens when you’re trying to set IP details on an adapter in a VMware VM that’s not connected.  We saw this on a few VMware VM’s that were recently stood up and had adapters but not connected to any networks.  Oops.

Potential Cause 2:

This is more of a delay/timing issue: While configuring a handful of VMware VM’s, I hit my stride with entering the IP information and found that when I entered the DNS IP address, tabbed down to the domain suffix and pressed [Enter] I’d get this error.  If I tried again another 1-2 seconds after getting the error it would accept.

Because I was running into this so often I spent a little time trying to test this and anecdotally it seems that when you make modifications to the domain suffix field, something happens behind the scenes that causes this delay.  I found that if I excluded the domain suffix field I didn’t have this problem at all.  I also learned that if I entered the domain suffix first then entered the DNS IP address I didn’t have this problem either.

It’s not clear what’s going on but I just need to ‘Trust But Verify’ the VM configuration and slow down when entering the details.  (^_^)

Good Providence to you!

MDT Tutorial Part 11: Troubleshooting Part 7: Non-Fatal OSD Errors & Warnings

Living Table of Contents

What These Guides Are:
A guide to help give you some insight into the troubleshooting process in general.

What These Guides Are Not:
A guide to fix all issues you’re going to encounter.

We’re going to role-play a bunch of scenarios and try to work through them.  Remember in math where you had to show your work?  Well, what follows is like that which is why this post is [more than] a [little] lengthy.

 

[Some] Non-Fatal OSD Errors & Warnings:

  • No networking adapters found, the network drivers for your device ar enot present

  • An invalid SLShareDynamicLogging value of <path> was specified

  • Unable to Create WebService class

You made a bunch of changes to your lab environment, tested the CS.INI before imaging but after performing a B&C you see these errors/warnings:

Yikes a bunch of errors here!

Yikes a bunch of errors here!

  • No networking adapters found, The network drivers for your device are not present
  • An invalid SLShareDynamicLogging value of \\ITF1MDT01\DeploymentShare$\TSLogs\BnC-22404895 was specified.
  • 2x Unable to create WebService class

To be sure, you delete the logs for that machine, run another B&C and confirm the BDD.log was indeed present and getting populated.  But at the end of the second B&C you received the same error and inspected the BDD.log further:

Troubleshoot-031.PNG

Starting from the top of the log:

  • The”Apply Windows PE” Task Sequence step completed successfully as evidenced by “Event 41001 sent: LTIApply processing completed successfully.”
    .
  • Then the “Add mass storage drivers to sysprep.inf for XP and 2003” Task Sequence step completed successfully as evidenced by “Event 41001 sent: ZTIDrivers processing completed successfully.”
    • Furthermore, the process is able to reach and write to \\ITF1MDT01\DeploymentShare$\TSLogs\BnC-22404895 so we know it was good at this point.
      This is important!
      .
  • After that the “Execute Sysprep” Task Sequence step completed successfully as evidenced by “Event 41001 sent: LTISysprep processing completed successfully.”
    • And that process was also able to reach and write to \\ITF1MDT01\DeploymentShare$\TSLogs\BnC-22404895.
      This is also important!
      .
  • Then we get to the “Apply Windows PE (BCD)” Task Sequence Step and almost as soon as it starts we see our errors:
    • No networking adapters found, The network drivers for your device are not present
    • An invalid SLShareDynamicLogging value of \\ITF1MDT01\DeploymentShare$\TSLogs\BnC-22404895 was specified.

From here you hypothesize that it’s the sysprep process that is creating this problem.
And like a wise man frequently tells me: We gotta peel this onion!

  • Found the error text in ZTIUtility.vbs: “An invalid SLShareDynamicLogging value”
  • Just above the error code it, two functions are called:
    • ValidateConnection
    • VerifyPathExists
      .
  • ValidateConnection
    • Is what drops the “Validating connection to” text in the log
    • Calls the ValidateNetworkConnectivity function which
      • Executes the query ​select * from win32_NetworkAdapter where Installed = true and adaptertypeid = 0" and evaluates the count of the results
      • If the result count is 0, the function exits after dropping “No networking adapters found, The network drivers for your device are not present” text in the log.
        .
  • VerifyPathExists checks the path of SLShareDynamicLogging, creating the directory if missing.  We know the directory exists because the files are where we expect them to be.
    .
  • Finally, the line that generates the “An invalid SLShareDynamicLogging” log entry is part of an if block, and is generated if the directory doesn’t exist.

With a better understanding of the process, we may have a good idea as to what might be happening:

  1. everything’s fine up until sysprep
  2. sysprep might be removing the NIC drivers and thus the adapter doesn’t work
  3. when the ‘Apply Windows PE (BCD)’ Task Sequence step (LTIApply.wsf) is executed immediately after sysprep, it fails to connect to the share because of Step 2 above
  4. once in WinPE the NIC is once again fully functional so everything works as expected

But what about the two Unable to create WebService class warning?  Thinking back to the recent changes you made, one of them was enabling Monitoring.   A little sleuthing might take you to a post by Michael Niehaus on Troubleshooting MDT 2012 Monitoring via a TechNet response by Johan Arwidmark.

After going through the guide, it still doesn’t work so you disable Monitoring, reimage, and the error goes away.  You re-enable Monitoring, image again, get the same result.

The good news is you’ve confirmed Monitoring is “to blame” for those errors.

A closer inspection of the BDD.log shows it’s part of the same LTIApply step it and that’s when it clicks: If there’s no network connectivity, there’s no way for the WebService to succeed.

At the very least, this is a plausible cause.

So for these errors and warnings, it probably “is what it is” and likely not that big of a deal in the grand scheme of things.

FWIW: I can reproduce this behavior with MDT8443 & ADK 1703 on Gen 2 and Gen 1 VM’s using both the Network Adapter and Legacy Network Adapter.  : (

Is it Possible to Fix All of These Errors?

Probably – in fact I’m fairly certain I have an older lab environment where I do not have this problem.  But there’s also something to be said about the return on investment and the law of diminishing returns.

If I can consistently reproduce something, and it really does create problems, I personally believe there’s value in looking into further.  It may be a typo or faux pas on my part or perhaps a valid ‘bug’ for a certain scenarios.

If it’s a one-off, something I see one in 30, it may not be worth investing a significant amount of time & effort for so little reward.

Copypasta Closing

Hopefully these examples will help give you an idea of the overall troubleshooting process.  Most of the time the problems you’ll encounter will be caused by a typso, order of operations or a ‘known issue’ that requires a specific process to be followed.

As you make changes to your environment, here’s what I recommend:

  • Be diligent about keeping a change log so you can easily backtrack
  • Backup your CS.INI or Bootstrap.ini before you make any changes
  • Backup your ts.xml or unattend.xml (in DeploymentShare\Control\TaskSequenceID) before you make any changes
  • Introduce small changes at time with set checkpoints in between and set milestones markers where you backup core files (e.g cs.ini bootstrap.ini ts.xml unattend.xml etc) to help minimize frustration troubleshooting.

And if when you do run into some turbulence, upload relevant logs (at least smsts.log but be prepared to submit others depending on the issue) to a file sharing service like OneDrive, post on TechNet then give a shout to your resources on Twitter.

Good Providence to you!

MDT Tutorial Part 11: Troubleshooting Part 6: Unable to mount the WIM, so the update process cannot continue

Living Table of Contents

 

What These Guides Are:
A guide to help give you some insight into the troubleshooting process in general.

What These Guides Are Not:
A guide to fix all issues you’re going to encounter.

We’re going to role-play a bunch of scenarios and try to work through them.  Remember in math where you had to show your work?  Well, what follows is like that which is why this post is [more than] a [little] lengthy.

Unable to mount the WIM, so the update process cannot continue

When updating your deployment share it fails almost immediately with error Unable to mount the WIM, so the update process cannot continue.

This slideshow requires JavaScript.

Error Text:


=== Making sure the deployment share has the latest x86 tools ===

=== Processing LiteTouchPE (x86) boot image ===

Building requested boot image profile.
Determining if any changes have been made in the boot image configuration.
No existing boot image profile found for platform x86 so a new image will be created.
Calculating hashes for requested content.
Changes have been made, boot image will be updated.
Windows PE WIM C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\x86\en-us\winpe.wim will be used.
Unable to mount the WIM, so the update process cannot continue.

=== Completed processing platform x86 ===

=== Making sure the deployment share has the latest x64 tools ===

=== Processing LiteTouchPE (x64) boot image ===

Building requested boot image profile.
Determining if any changes have been made in the boot image configuration.
No existing boot image profile found for platform x64 so a new image will be created.
Calculating hashes for requested content.
Changes have been made, boot image will be updated.
Windows PE WIM C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\en-us\winpe.wim will be used.
Unable to mount the WIM, so the update process cannot continue.

=== Completed processing platform x64 ===

=== Processing complete ===

You might try a variety of things like:

  • Use DISM to get-wiminfo to confirm the WIMs are OK
  • Bounce the MDT machine
  • Perform a repair installation of the ADK
  • Use DISM to mount the WIM manually and you get an error 577

We talked about this in Part 2 : Did you remember to patch the ADK?  🙂

Copypasta Closing

Hopefully these examples will help give you an idea of the overall troubleshooting process.  Most of the time the problems you’ll encounter will be caused by a typso, order of operations or a ‘known issue’ that requires a specific process to be followed.

As you make changes to your environment, here’s what I recommend:

  • Be diligent about keeping a change log so you can easily backtrack
  • Backup your CS.INI or Bootstrap.ini before you make any changes
  • Backup your ts.xml or unattend.xml (in DeploymentShare\Control\TaskSequenceID) before you make any changes
  • Introduce small changes at time with set checkpoints in between and set milestones markers where you backup core files (e.g cs.ini bootstrap.ini ts.xml unattend.xml etc) to help minimize frustration troubleshooting.

And if when you do run into some turbulence, upload relevant logs (at least smsts.log but be prepared to submit others depending on the issue) to a file sharing service like OneDrive, post on TechNet then give a shout to your resources on Twitter.

Good Providence to you!

Deployment Error Invalid DeploymentType value "" specified.  The deployment will not proceed.

MDT Tutorial Part 11: Troubleshooting Part 5: Invalid DeploymentType value “” specified. The deployment will not proceed.

Living Table of Contents

 

What These Guides Are:
A guide to help give you some insight into the troubleshooting process in general.

What These Guides Are Not:
A guide to fix all issues you’re going to encounter.

We’re going to role-play a bunch of scenarios and try to work through them.  Remember in math where you had to show your work?  Well, what follows is like that which is why this post is [more than] a [little] lengthy.

Invalid DeploymentType value “” specified. The deployment will not proceed.

You enabled the Windows Updates steps in the Build & Capture Task Sequence so you can have a fully patched Windows 10 v1511 WIM.

Troubleshoot-014.PNG

And you also updated your CustomSettings.ini so that your Build & Capture VM would set all the properties/variables but not start immediately.


[Settings]
Priority=MACAddress,GetAbbrModel,Build,Default
Properties=OfficeCode,AbbrModel

;vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
; BEGIN MACADDRESS SECTION
; This is my Windows 10 v1511 build & capture VM
[00:15:5D:13:79:01]
SkipTaskSequence=NO
TaskSequenceID=BC151164ENT
SkipComputerName=NO
OSDComputerName=BnC-#UCase(Right(Replace(Replace("0000000%SERIALNUMBER%"," ","",1,-1,1),"-","",1,-1,1),8))#
SkipDomainMembership=NO
JoinWorkgroup=BnC-WrkGrp
SkipUserData=NO
SkipComputerBackup=NO
ComputerBackupLocation=NETWORK
BackupDir=Captures\%OSDComputerName%
BackupFile=%OSDComputerName%_%TaskSequenceID%_#year(date) & "-" & month(date) & "-" & day(date) & "_" & Hour(Now()) & Minute(Now())#.wim
SkipProductKey=NO
SkipLocaleSelection=NO
SkipTimeZone=NO
SkipAdminPassword=NO
SkipCapture=NO
DoCapture=YES
SkipBitLocker=NO
SkipSummary=NO

; END MACADDRESS SECTION
;^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

;vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
; BEGIN GETABBRMODEL SECTION
; Lets get the abbreviated model
[GetAbbrModel]
UserExit=jgp_GetAbbrModel.vbs
AbbrModel=#GetAbbrModel#
; END GETABBRMODEL SECTION
;^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

;vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
; BEGIN BUILD SECTION
; Set things here for use below
[Build]
OSDComputerName=%OfficeCode%-%AbbrModel%-#UCase(Right(Replace(Replace("0000000%SERIALNUMBER%"," ","",1,-1,1),"-","",1,-1,1),8))#
; END GETABBRMODEL SECTION
;^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

;vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
; BEGIN DEFAULT SECTION
[Default]
OSInstall=Y
; Skip Screen: Task Sequence
SkipTaskSequence=NO
; Skip Screen: Computer Details
SkipComputerName=NO
; Skip Screen: Computer Details
SkipDomainMembership=NO
; Skip Screen: Move Data and Settings & User Data (Restore)
SkipUserData=NO
; Skip Screen: Computer Backup
SkipComputerBackup=NO
; Skip Screen: Product Key
SkipProductKey=NO
; Skip Screen: Locale & Time
SkipLocaleSelection=NO
KeyboardLocale=en-US
UserLocale=en-US
UILanguage=en-US
; Skip Screen: Locale & Time
SkipTimeZone=NO
; https://msdn.microsoft.com/en-us/library/ms912391(v=winembedded.11).aspx
TimeZoneName=Eastern Standard Time
; Skip Screen: Administrator Password
SkipAdminPassword=NO
; Skip Screen: Capture Image
SkipCapture=NO
; Skip Screen: BitLocker
SkipBitLocker=NO
; Skip Screen: Ready to begin
SkipSummary=NO
; Skip Screen: R
SkipFinalSummary=NO
SLShare=%DeployRoot%\TSLogs
SLShareDynamicLogging=%DeployRoot%\TSLogs\%OSDComputerName%
EventService=http://ITF1MDT01:9800
; END DEFAULTSECTION
;^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Having learned your lesson from last time, you test the updated CustomSettings.ini & spent a few minutes verifying the output in the console.  Feeling confident, you boot into WinPE and when the Wizard I displays, verify all the defaults are set correctly.

This slideshow requires JavaScript.

You click begin and almost immediately the process halts with an obscure error:

Deployment Error Invalid DeploymentType value

Deployment Error Invalid DeploymentType value “” specified.  The deployment will not proceed.

You’d be forgiven for thinking you’ve made a mistake and I’d be willing to bet you’d figure this out but that road wouldn’t be fun.

Fortunately Michael Niehaus himself confirmed this is a bug (SRC1, SRC2) and there are two options to fixing/working around the bug:

  1. Easiest: Update your CS.INI with SkipProductKey=YES.
  2. Hardest (and arguably unsupported?):
    1. Open DeployWiz_ProductKeyVista.vbs in the Scripts directory
    2. Find: ​if oProperties("DeploymentType") = "UPGRADE" then
    3. Replace: If Property("DeploymentType") = "UPGRADE" then
    4. Save the file and try again
Note: There appears to be a plausible explanation of what’s happening here if you’re interested.

Since you rarely need to prompt for a Product Key – especially considering you can stick it in the unattend.xml or you go down the easy path and now the image kicks off.

Troubleshoot-028.PNG

Copypasta Closing

Hopefully these examples will help give you an idea of the overall troubleshooting process.  Most of the time the problems you’ll encounter will be caused by a typso, order of operations or a ‘known issue’ that requires a specific process to be followed.

As you make changes to your environment, here’s what I recommend:

  • Be diligent about keeping a change log so you can easily backtrack
  • Backup your CS.INI or Bootstrap.ini before you make any changes
  • Backup your ts.xml or unattend.xml (in DeploymentShare\Control\TaskSequenceID) before you make any changes
  • Introduce small changes at time with set checkpoints in between and set milestones markers where you backup core files (e.g cs.ini bootstrap.ini ts.xml unattend.xml etc) to help minimize frustration troubleshooting.

And if when you do run into some turbulence, upload relevant logs (at least smsts.log but be prepared to submit others depending on the issue) to a file sharing service like OneDrive, post on TechNet then give a shout to your resources on Twitter.

Good Providence to you!

MDT Tutorial Part 11: Troubleshooting Part 4: Task Sequence Variable is Being Overwritten

Living Table of Contents

 

What These Guides Are:
A guide to help give you some insight into the troubleshooting process in general.

What These Guides Are Not:
A guide to fix all issues you’re going to encounter.

We’re going to role-play a bunch of scenarios and try to work through them.  Remember in math where you had to show your work?  Well, what follows is like that which is why this post is [more than] a [little] lengthy.

Task Sequence Variable is Being Overwritten – The Case of the Incorrectly Named Log Folder

After [finally] fixing your computer naming issue, you decide to take a look at the live BDD.log just to keep an eye on things.  You hop into the TSLogs directory & find 2 directories that have the same time stamp:

Troubleshoot-008

According to your CS.INI the “real” log directory should be named after the machine so how did that other one get created?  You open the BDD.log in BLD-HPV-22404895 and it’s practically empty, but you do see that the property is being set to the other directory.

Troubleshoot-009.PNG

There are some clues in the BDD that lead you to above that point to a smoking gun.

You open the BDD.log in the other directory & scroll to the top & see a familiar set of lines: The same ones you see when testing the CustomSettings.ini manually:

Troubleshoot-011

While the machine is imaging you press F8 to launch a console, fire up CMTrace, open the smsts.log (in %temp%\smstslog), scroll to the top & search downwards for ‘dyn’:

Troubleshoot-012.PNG

Ok, you’re piecing it together:

  1. Initially it’s good, points to the proper location
  2. Then at some point it changes
  3. In the first, short, BDD.log you see ‘Task Sequence’ mentioned
  4. In the second, long, BDD.log you see lines that look like it’s processing the CustomSettings.ini, something that also happens during the Task Sequence,
  5. The smsts.log shows a step that sets a variable
  6. SLShareDynamicLogging isn’t a property (or variable) that is ‘last write wins’, so it can’t be a case of a rogue CustomSettings.ini

So maybe it’s the Task Sequence?

Troubleshoot-013

Bingo was his name.

 

Copypasta Closing

Hopefully these examples will help give you an idea of the overall troubleshooting process.  Most of the time the problems you’ll encounter will be caused by a typso, order of operations or a ‘known issue’ that requires a specific process to be followed.

As you make changes to your environment, here’s what I recommend:

  • Be diligent about keeping a change log so you can easily backtrack
  • Backup your CS.INI or Bootstrap.ini before you make any changes
  • Backup your ts.xml or unattend.xml (in DeploymentShare\Control\TaskSequenceID) before you make any changes
  • Introduce small changes at time with set checkpoints in between and set milestones markers where you backup core files (e.g cs.ini bootstrap.ini ts.xml unattend.xml etc) to help minimize frustration troubleshooting.

And if when you do run into some turbulence, upload relevant logs (at least smsts.log but be prepared to submit others depending on the issue) to a file sharing service like OneDrive, post on TechNet then give a shout to your resources on Twitter.

Good Providence to you!

Troubleshooting the Lenovo ThinInstaller Installation

During OSD, install the Lenovo ThinInstaller, and have been doing this for quite some time.  Whenever a new build is released, we duplicate the source files and swapping out the older installer for the new.  This process worked well for several versions:

  • v1.2.0014
  • v1.2.0015
  • v1.2.0017
  • v1.2.0018

In early 2017 we updated our Lenovo ThinInstaller to v1.2.0020 and noticed that the ThinInstaller installation executable would run seemingly forever, but during OSD and in real Windows when installing manually.  We reverted back to 1.2.0018 until we could do more testing to narrow the scope of the problem.  Shortly afterwards, 1.2.0022 was released so we tried again thinking maybe it was a build-specific bug, but it too failed.

We spent a decent amount of time trying to hunt this down and confirmed it wasn’t:

  • unique to the Task Sequence
  • a model specific issue
  • an upgrade scenario (e.g.: only 1.2.0014 to 1.2.0022)
  • software that was already installed and causing a conflict
  • security software on the machine.

Again, we could reproduce at will by removing the new installer and restoring the old.

A while later we learned that Lenovo had made some changes to the ThinInstaller package, and you can read about that here:
https://thinkdeploy.blogspot.com/2017/03/changes-required-to-use-thin-installer.html

To ensure the highest rate of installation success, we took the shotgun approach:


# Unregister the DLL if it's found - update the path accordingly
if(Test-Path -Path "$envProgramFilesX86\ThinInstaller\Lenovo.PnPSignedDriverEx.dll" -PathType Leaf)
    {
        If(Test-Path -Path "$env:windir\Microsoft.NET\Framework\v4.0.30319\InstallUtil.exe" -PathType Leaf)
            {
                Start-Process -FilePath "$env:windir\Microsoft.NET\Framework\v4.0.30319\InstallUtil.exe" -ArgumentList "/u `"$envProgramFilesX86\ThinInstaller\Lenovo.PnPSignedDriverEx.dll`"" -PassThru -Wait
            }
        elseif(Test-Path -Path "$env:windir\Microsoft.NET\Framework\v2.0.50727\InstallUtil.exe" -PathType Leaf)
            {
                Start-Process -FilePath "$env:windir\Microsoft.NET\Framework\v2.0.50727\InstallUtil.exe" -ArgumentList "/u `"$envProgramFilesX86\ThinInstaller\Lenovo.PnPSignedDriverEx.dll`"" -PassThru -Wait
            }
    }

# Remove the existing 'installation'
Rename-Item -Path "${env:ProgramFiles(x86)}\ThinInstaller" -NewName "ThinInstaller_$(Get-Date -Format 'yyyy-MM-dd_hhmmss')"; Start-Sleep -Seconds 3

# Perform the 'install'
Start-Process -FilePath 'thin_installer_VERSION.exe' -ArgumentList "/SP- /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /LOG=Path\To\Install.log" -Wait

# Drop in the ThinInstaller.exe.configuration
Copy-Item -Path $(Join-Path $PSScriptRoot 'ThinInstaller.exe.configuration') -Destination "${env:ProgramFiles(x86)}\ThinInstaller\ThinInstaller.exe.configuration" -Force&lt;span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			&gt;&amp;#65279;&lt;/span&gt;

 

Some time around ThinInstaller v1.2.0029, the Lenovo.PnPSignedDriverEx.dll as well as the .cmd files used to register/unregister said DLL, quietly disappeared suggesting they either found a better way to do what they were doing or baked that process into their installer or … ?  That said, the above code is probably no longer needed.  However, since ThinInstaller is an application that’s generally available to all Lenovo machines in Software Center, we left the code in there on the off chance someone upgrades or uninstalls/reinstalls ThinInstaller on a machine with an older build.

If you find the ThinInstaller is hanging during upgrade scenarios, try the above to see if that helps.

If you find the ThinInstaller is hanging during a fresh installation, try generating a log and review the log file to see where it’s getting hung up then troubleshoot from there.

Good Providence To You!

Title: Windows Setup Body: Windows could not parse or process unattend answer file [C:windowsPantherunattend.xml] for pass [specialize]. The answer file is invalid.

MDT Tutorial Part 11: Troubleshooting Part 3: Windows could not parse or process unattend answer file [C:\windows\Panther\unattend.xml] for pass [specialize].  The answer file is invalid.

Living Table of Contents

 

What These Guides Are:
A guide to help give you some insight into the troubleshooting process in general.

What These Guides Are Not:
A guide to fix all issues you’re going to encounter.

We’re going to role-play a bunch of scenarios and try to work through them.  Remember in math where you had to show your work?  Well, what follows is like that which is why this post is [more than] a [little] lengthy.

Windows could not parse or process unattend answer file [C:\windows\Panther\unattend.xml] for pass [specialize].  The answer file is invalid.

Your last victory is short lived as the same error message appears and this time unattend.xml looks fine:

Troubleshoot-010.PNG

Stumped, you might search for ‘Microsoft-Windows-Shell-Setup’ which might lead you here:
https://docs.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-shell-setup

As you review each section carefully the issue becomes clear: The computer name is more than 15 characters.

Copypasta Closing

Hopefully these examples will help give you an idea of the overall troubleshooting process.  Most of the time the problems you’ll encounter will be caused by a typso, order of operations or a ‘known issue’ that requires a specific process to be followed.

As you make changes to your environment, here’s what I recommend:

  • Be diligent about keeping a change log so you can easily backtrack
  • Backup your CS.INI or Bootstrap.ini before you make any changes
  • Backup your ts.xml or unattend.xml (in DeploymentShare\Control\TaskSequenceID) before you make any changes
  • Introduce small changes at time with set checkpoints in between and set milestones markers where you backup core files (e.g cs.ini bootstrap.ini ts.xml unattend.xml etc) to help minimize frustration troubleshooting.

And if when you do run into some turbulence, upload relevant logs (at least smsts.log but be prepared to submit others depending on the issue) to a file sharing service like OneDrive, post on TechNet then give a shout to your resources on Twitter.

Good Providence to you!

Title: Windows Setup Body: Windows could not parse or process unattend answer file [C:windowsPantherunattend.xml] for pass [specialize]. The answer file is invalid.

MDT Tutorial Part 11: Troubleshooting Part 2: Windows could not parse or process unattend answer file [C:\windows\Panther\unattend.xml] for pass [specialize].  The answer file is invalid.

Living Table of Contents

 

What These Guides Are:
A guide to help give you some insight into the troubleshooting process in general.

What These Guides Are Not:
A guide to fix all issues you’re going to encounter.

We’re going to role-play a bunch of scenarios and try to work through them.  Remember in math where you had to show your work?  Well, what follows is like that which is why this post is [more than] a [little] lengthy.

Windows could not parse or process unattend answer file [C:\windows\Panther\unattend.xml] for pass [specialize].  The answer file is invalid.

You boot your special VM, click the ‘Run the Deployment Wizard to install a new Operating System‘ button and it immediately starts.  Excellent!  It applies the OS, reboots and you’re faced with this error:

Title: Windows Setup Body: Windows could not parse or process unattend answer file [C:\windows\Panther\unattend.xml] for pass [specialize].  The answer file is invalid.

Windows could not parse or process unattend answer file [C:\windows\Panther\unattend.xml] for pass [specialize]. The answer file is invalid.

Well this is strange, because you didn’t touch the unattend.xml so what gives?
Fortunately, this dialog provides some meaningful insight:

    • The unattend file is C:\Windows\Panther\unattend.xml
    • The specific area is the specialize pass

Press SHIFT+F10 here to open a command prompt and then open C:\Windows\Panther\unattend.xml with notepad

Troubleshoot-005

You search for ‘specialize’ and after taking a very close look see that your computer name is incorrect.  It should be some two or three character prefix not %OfficeCode%.

Troubleshoot-006

Since that is set via the CS.INI, you run the CustomSettings.ini test again and now you see what was missed before:

Troubleshoot-007.PNG

You review the CS.INI and find your problems

  1. You didn’t define the OfficeCode property: Wasn’t added to the Properties line
  2. You didn’t set a value for OfficeCode.

With that fixed, you run the test again, the variable is populated and as you reimage the machine, you see it is named correctly in the logs.

Copypasta Closing

Hopefully these examples will help give you an idea of the overall troubleshooting process.  Most of the time the problems you’ll encounter will be caused by a typso, order of operations or a ‘known issue’ that requires a specific process to be followed.

As you make changes to your environment, here’s what I recommend:

  • Be diligent about keeping a change log so you can easily backtrack
  • Backup your CS.INI or Bootstrap.ini before you make any changes
  • Backup your ts.xml or unattend.xml (in DeploymentShare\Control\TaskSequenceID) before you make any changes
  • Introduce small changes at time with set checkpoints in between and set milestones markers where you backup core files (e.g cs.ini bootstrap.ini ts.xml unattend.xml etc) to help minimize frustration troubleshooting.

And if when you do run into some turbulence, upload relevant logs (at least smsts.log but be prepared to submit others depending on the issue) to a file sharing service like OneDrive, post on TechNet then give a shout to your resources on Twitter.

Good Providence to you!