question

MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 asked BillStewart commented

Different formatting

Hello,

Strange problem: I have a script that reads an object's ACL and saves it to a text file (Get-Acl objectname -Audit |FL > "textfile.txt"). Script has been working fine until I've bumped into the weird issue - occasionally the format of the resulting txt file is different from the one created previously (as I need to compare the old and the new files this leads to the false triggering), for example, if I run the script 6 times in a raw the format of the Sddl section will look as in the blue rectangle, but the 7th will look like in the red:

46494-q3.png

It does not matter whether there were any changes in ACLs or not - the format of SDDL strings can change without any (at least noticable) reason.

The two files do differ in formatting...
46651-q1.png

...but not in the content:
46661-q2.png



I've spent more than two days trying to pinpoint the situation in which those changes occur but to no avail, I just found out that the format may change any time in any way: it can change during the last 7 minutes or only the day after the script was run for the first time. The Sddl section may be made of 4-5-6 short strings, may be of 3-4 long ones. What can affect the way the PS formats a file???

Thank you in advance,
Michael

windows-server-powershellwindows-server-2019windows-server-2016
q3.png (31.5 KiB)
q1.png (19.2 KiB)
q2.png (60.2 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

RichMatheisen-8856 avatar image
0 Votes"
RichMatheisen-8856 answered

The first thing I'd suggest is that you not save the output of Format-List to a text file. The second is that without seeing the code you're using it's not possible to do more than guess at the reason for the differences.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 answered RichMatheisen-8856 edited

"that you not save the output of Format-List to a text file" - sorry - did not understand it.

Here's the code: Get-Acl C:\test\Testfile.txt -Audit |FL > "c:\Temp\1.txt"

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

"FL" is an alias for Format-List. That cmdlet is intended for displaying the object in a list format. It's output shouldn't be sent to a file.

A better way would be to use Select-Object. Assuming you're interested in a property of the ACL named "Sddl", try it this way:

 Get-Acl C:\test\Testfile.txt -Audit | Select-Object -Expand Sddl | Out-File c:\Temp\1.txt
0 Votes 0 ·
MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 answered RichMatheisen-8856 commented

"That cmdlet is intended for displaying the object in a list format. It's output shouldn't be sent to a file." - no, this cmdlet (as any other format cmdlets) is intended just for arranging the data - not for displaying it:

For example, this article says:
The format cmdlets, such as Format-List,
arrange the data to be displayed but do not display it. The data is displayed by the output features of PowerShell and by the cmdlets that contain the Out verb (the Out cmdlets), such as Out-Host or Out-File
.
If you do not use a format cmdlet, PowerShell applies that default format for each object that it displays.


Having read the theory above why should one not use ...|FL > somefile ?


In any case, it's my understanding that it's not normal that the same procedure yields different results.


If you could post a link to such statement - It's output shouldn't be sent to a file - I'd very much appreciate it.








· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Don't confuse shouldn't with can't. You're free to do as you wish. What you quoted is correct, Format-List arranges the data there's no argument. Whether that formatting is appropriate to your intended use is the issue. If you're concerned only with comparing the Sddl property of the ACL then that's what should be in the file you're creating and then comparing. As you note, the contents of the files are identical. Only the formatting differs.

So, what the first thing that you notice when you look at the two different formats? The first thing that catches my eye is that the length of the lines are different. Are the widths of the windows in which you're running #1 thru #6 different to the one in which you're running #7? Remember, Format-List and Format-Table arrange the content for display.


0 Votes 0 ·
MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 answered BillStewart commented

P.S. Here's the example of FL > file:

https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/format-list?view=powershell-7.1

Notes

Input objects are automatically formatted as they would be in the terminal, but you can use a Format- cmdlet to explicitly control the formatting of the output to the file. For example, Get-Date | Format-List | Out-File out.txt*


· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I think what you really want is to get the SDDL property and export that, as I don't think that list format's output is contractual (the "explicitly control the formatting" means "you get a list format", not "the list format is guaranteed to be the same all the time").

As Rich said, you will probably get better and more consistent results if you do this:

 Get-Acl C:\test\Testfile.txt -Audit | Select-Object -ExpandProperty Sddl | Out-File c:\Temp\1.txt
0 Votes 0 ·
MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 answered BillStewart edited

Hello,
Thank you all for your replies!


Whether that formatting is appropriate to your intended use is the issue. - I'd say the issue is this: "not "the list format is guaranteed to be the same all the time""

"Are the widths of the windows in which you're running #1 thru #6 different to the one in which you're running #7" - yes, they are. The problem arises when the new file has extra Carriage return/EOL... characters (or on contrary, does not have).

And I still don't understand why it should be better (theoretically):

"As Rich said, you will probably get better and more consistent results if you do this:"

Get-Acl C:\test\Testfile.txt -Audit | Select-Object -ExpandProperty Sddl | Out-File c:\Temp\1.txt

As MS says -
The data is displayed by the output features of PowerShell and by the cmdlets that contain the Out verb (the Out cmdlets), such as Out-Host or Out-File. - accrding to this any extra cmdlets (such as FL, FT) should not affect the way the file will look like - only PS itself + Out cmdlets must define the formatting.

I'll try to do more testing and post back the results.

Regards,
Michael

P.S.

If "(the "explicitly control the formatting" means "you get a list format", not "the list format is guaranteed to be the same all the time")" - if specifying the same set of parameters does not lead to the same output I would not call that process "explicitly control the formatting" :(

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

if specifying the same set of parameters does not lead to the same output I would not call that process "explicitly control the formatting" :(

I think it would help if you view formatted output as convenient for viewing but not for further processing. Another way to say this is: If you want to do something with the data, select the properties you want directly (don't use the formatting cmdlets for that use case).


0 Votes 0 ·
MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 answered

...the last point... :

47424-11.png

47444-12.png



...it just does not work.

Thank you all for your help!

Regards,
Michael


11.png (56.1 KiB)
12.png (48.6 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 answered

P.S.

And the cherry on the cake:

47378-13.png




As you see on the screenshot above the files were copied to C:\111 (and first compared) at 2:21 PM. I've been running this code since that time ~ 10 times (at ~3:00PM, 3:30PM, 16:05, ...18:03PM), the result was False. At 18:07 the result became True. The question is closed.


13.png (121.3 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MikhailFirsov-1277 avatar image
0 Votes"
MikhailFirsov-1277 answered BillStewart commented

Hi BillStewart,

"I think it would help if you view formatted output as convenient for viewing but not for further processing. Another way to say this is: If you want to do something with the data, select the properties you want directly (don't use the formatting cmdlets for that use case)." - I guess you're right.

But it means that this -
"The format cmdlets, such as Format-List, arrange the data to be displayed but do not display it. The data is displayed by the output features of PowerShell and by the cmdlets that contain the Out verb (the Out cmdlets), such as Out-Host or Out-File. "

  • is not true because - according to the statements above - arranging the data should not affect the "the output features of PowerShell", especially when you run the same command several times.

Regards,
Michael



· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

But it means that this -
"The format cmdlets, such as Format-List, arrange the data to be displayed but do not display it. The data is displayed by the output features of PowerShell and by the cmdlets that contain the Out verb (the Out cmdlets), such as Out-Host or Out-File."
is not true because - according to the statements above - arranging the data should not affect the "the output features of PowerShell", especially when you run the same command several times.

I don't see where it says that the output features of PowerShell will guarantee that formatted output will be formatted identically at every invocation, so I guess I would say that my understanding of the statements you listed is different from yours.

0 Votes 0 ·

Seems that "the output features of PowerShell" does guarantee the same format on each invocation, it is the FORMATTING cmdlets that DO NOT. And this is what I call "not correct" because by defenition above the formatting cmdlets should NOT affect the way the default "output features of PowerShell" work: they should NOT - but they DO!

0 Votes 0 ·

Seems that "the output features of PowerShell" does guarantee the same format on each invocation, it is the FORMATTING cmdlets that DO NOT.

Cmdlet output without piping to a formatting cmdlet is (by definition) not formatted, so I guess I would disagree with this definition. I would consider the possibility that you may be misunderstanding the documentation.

In any case, I think you have an answer and explanation at this point.

0 Votes 0 ·