Tag: Filter

  • How to filter for PowerShell objects easily

    If you are working interactively with PowerShell, It might be unhandy to define all the filtering options with Where-Object. I will show you how to filter for PowerShell objects the traditional way and doing it with Out-Gridview.

    Filter for PowerShell objects – the traditional way with Where-Object

    If we are looking forward to filter in PowerShell, we normaly make use of Where-Object this. In my example I am looking forward for all proccesses, which contain the name msedge and the value for Handles is greater than 292. My query would look like this:

    #traditional way
    Get-Process  | Where-Object {$_.processname -contains "msedge" -and $_.Handles -gt 292}
    Output of all processes in PowerShell
    Get-Process the traditional way with Where-Object

    Filter for PowerShell objects – Out-Gridview

    I want to show you how to filter for Powershell objects easily by using following cmdlet:

    Out-GridView -PassThru

    Example:

    Get-Process | Out-GridView -PassThru

    Doing that results in a grid popping up.

    output of processes as a grid in PowerShell

    Now you have multiple options.

    You can make use of the filter of the out of the box filtering options. Let’s say I just want to find the msedge processes.

    So I am adding the Processname as a filter criteria:

    filtering options in the grid in PowerShell

    Adding ProcessName eligibles setting filtering following operators:

    contains
    does not contains
    beginns with
    is equal to
    is not equal to
    ends with
    is empty
    is not empty

    Entering msedge to the contains field, shows only process entries, which contain msedge

    filtered results in PowerShell grid
    filtering for processes which contain msedge in the name

    Now you can add additional filters.

    Additional filters in PowerShell
    I do add NPM(K) for the value 19
    Filter for npmk equals 19

    We can even now sort all the stuff by clicking on the column, which we want to sort for.

    Sort for CPU in GUI
    Sorting for highest CPU usage

    Export objects to Excel

    If you mark everything with CTRL + A, you can copy all the stuff and put it e.g. straigth to Excel

    Exporting objects to Excel
    Easy way to export to excel

    If you want to process further with powershell, you can mark the entities needed and click on OK. I selected the first and the thridh entity.

    Picture selecting only two entities
    selecting only two entities

    As you see, PowerShell returns my selection

    PowerShell return

    Conclusio

    As you can see you can save time and coding lines, when use are working inteactively on PowerShell by filter forPowerShell objects with Out-Gridview.

    You might find intersting

    Filtering items with CAML. For further learnings check out the post: Filtering for SharePoint items with CAML Queries | SPO Scripts

    Official documentation of Microsoft for the cmdlet Out-Gridview: Out-GridView (Microsoft.PowerShell.Utility) – PowerShell | Microsoft Docs

  • How to use LINQ in PowerShell to compare arrays

    How to use LINQ in PowerShell to compare arrays

    LINQ stands for Language Integrated Query and if you use LINQ in PowerShell it might boost the performance of your scripts. In this article, I’ll show you how to use LINQ in PowerShell for comparing arrays.

    In this article, I will show you except and intersect of LINQ. There are many more methods. If you are interested in other methods, check out the official documentation of Microsoft.

    Concept of using LINQ in PowerShell

    We have three amounts red, orange and yellow. The aim is to only get a specific amount e.g. only the red portion.

    Amounts with strings
    $Red = @( "A", "B", "C")
    $Yellow = @("C","D","E")

    We also can do this for integer values.

    amounts with integers

    $Red = @( 1..5)
    $Yellow = @(4..10)

    Using LINQ for Strings in arrays

    I will show you how to use LINQ for string in this chapter. We will cover the methods except and intersect.

    String – Get the red amount

    In order to get the red amount, you have to do following:

    $Red = @( "A", "B", "C")
    $Yellow = @("C","D","E")
    
    $Left  = [string[]]$Red
    $Right = [string[]]$Yellow
    
    [string[]][Linq.Enumerable]::Except($Left, $Right)

    By doing this, you’ll get only A and B:

    LINQ get the red amount in PowerShell

    String – Get the yellow amount

    In order to get the red amount, you have to change Right and Left in the LINQ cmdlet:

    $Red = @( "A", "B", "C")
    $Yellow = @("C","D","E")
    
    $Left  = [string[]]$Red
    $Right = [string[]]$Yellow
    
    [string[]][Linq.Enumerable]::Except($Right, $Left)

    By doing this, you’ll get only D and E:

    LINQ get the yellow amount in PowerShell

    String – Get the orange amount

    If we want to get the orange amount, we have to make use of intersect

    $Red = @( "A", "B", "C")
    $Yellow = @("C","D","E")
    
    $Left  = [string[]]$Red
    $Right = [string[]]$Yellow
    
    [string[]][Linq.Enumerable]::Intersect($Left, $Right)
    LINQ get the orange amount in PowerShell

    String – Get everything except orange

    $Red = @( "A", "B", "C")
    $Yellow = @("C","D","E")
    
    $Left  = [string[]]$Red
    $Right = [string[]]$Yellow
    
    [string[]]([Linq.Enumerable]::Except($Left, $Right) + [Linq.Enumerable]::Except($Right, $Left))

    By doing this, you’ll everything but not C:

    linq - get everything except the orange amount powershell

    Using LINQ for integers in arrays

    I will show you how to use LINQ for integers in this chapter. We will cover the methods except and intersect.

    Integer – Get the red amount

    In order to get the red amount, you have to do following:

    $Red = @( 1..5)
    $Yellow = @(4..10)
    
    $Left  = [int[]]$Red
    $Right = [int[]]$Yellow
    
    [int[]][Linq.Enumerable]::Except($Left, $Right)

    By doing this, you’ll get only 1, 2, 3:

    LINQ get only the red amount in PowerShell

    Integer – Get the yellow amount

    In order to get the red amount, you have to change Right and Left in the LINQ cmdlet:

    $Red = @( 1..5)
    $Yellow = @(4..10)
    
    $Left  = [int[]]$Red
    $Right = [int[]]$Yellow
    
    [int[]][Linq.Enumerable]::Except($Right, $Left)

    By doing this, you’ll get only 6, 7, 8, 9, 10:

    LINQ get the yellow amount in PowerShell for integers

    Integer – Get the orange amount

    If we want to get the orange amount, we have to make use of intersect

    $Red = @( 1..5)
    $Yellow = @(4..10)
    
    $Left  = [int[]]$Red
    $Right = [int[]]$Yellow
    
    [int[]][Linq.Enumerable]::Intersect($Right, $Left)
    LINQ get the orange amount in PowerShell for integers

    Integer – Get everything except orange

    $Red = @( 1..5)
    $Yellow = @(4..10)
    
    $Left  = [int[]]$Red
    $Right = [int[]]$Yellow
    
    [int[]]([Linq.Enumerable]::Except($Left, $Right) + [Linq.Enumerable]::Except($Right, $Left))

    By doing this, you’ll everything but not 5:

    LINQ get the everything but not the orange amount in PowerShell for integers
    Read more
  • Filtering for SharePoint items with CAML Queries

    Filtering for SharePoint items with CAML Queries

    Most of our times, we just need just a bunch of items, to export them or to change their values. This post should help you to show, how to handle filtering for SharePoint items. Besides filtering for SharePoint items with Where-Object, you can also make use of CAML (Collaborative Application Markup Language), which lets you get only the items, you need. It might increase the performance of your queries, when you are dealing with large amounts of data.

    Where are the items, which I am looking for?

    Preqrequistes

    If we want to achieve filtering for SharePoint items, with a CAML query, we have to fulfill following prerequisites:

    1. Permissions to access the list
    2. Installed Module PNP.Powershell. If you don’t know how to, check the post.
    3. Connection to the site via PNP.PowerShell. If you don’t know how to, check the post.

    Considerations

    1. You should take care of the case sensitivity of operands and column names
    2. You should take care of the <view> part. Sometimes it is needed, sometimes not – so I would rely on the examples.

    Query Schema

    A query is structured like this

     "<View><Query><Where><LOGICAL OPERATOR><FieldRef Name='INTERNAL NAME OF COLUMN'/><Value Type='VALUE TYPE'>VALUE</Value></LOGICAL OPERATOR></Where></Query></View>"

    You can find the internal name of columns in two ways:

    PowerShell or GUI.

    Explanation for PowerShell: Getting FieldValues of Items | SPO Scripts
    Explanation for GUI: Determine internal name of SharePoint Columns with GUI (workplace-automation.com/)

    Value types

    TypeMeaningExamples
    BooleanIt means true or false. You can find this in yes/no checkboxestrue, false
    true reflects 1
    false reflects 0
    ChoiceIt reflects the choices in your sharepoint listapple, banana
    CurrencyIt reflects the amount of an defined currency5$
    DateTimeIt reflects a timestamp23.06.2021 15:30
    GUIDGlobally Unique Identifier (GUID)6154ff96-8209-457b-86dd-ee7dcd80b584
    IntegerIt reflects a number as an integer 10
    LookupLinks to another list: for example an Orders list may have a lookup field that links to customers in a Customer list;Füller AG
    NoteReflects a multi line text field. Not sortable or groupableLorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec odio.

    Quisque volutpat mattis eros. Nullam malesuada erat ut turpis. Suspendisse urna nibh, viverra non, semper suscipit, posuere a, pede.
    TextReflects a single line text field. Sortable and groupable. Corresponds to the nvarchar SQL data type and represented by the SPFieldText class.Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
    UserA Lookup field that references the UserInfo database table.Email TypeId LookupId LookupValue
    —– —— ——– ———–
    Serkar@devmodernworkplace.onmicrosoft.com {c956ab54-16bd-4c18-89d2-996f57282a6f} 11 Serkar Aydin
    Source: Field element (Field) | Microsoft Docs

    Logical Comparison Operators

    In this case X means your entry

    OperatorMeaning
    BeginsWithThe existing value begins with X
    ContainsThe existing value contains x
    DateRangesOverlapThe existing date overlaps the date range defined in x
    |---- 01.01.-07.01 ------|
    |---02.01-09.01 -----|
    EqThe existing value equals x
    GeqThe existing value is greater or equal x
    GtThe existing value is greater than x
    InX is one of the existing values
    Includeschecks, whether x is in the defined values
    NotIncluseschecks, whether x is not in the defined values
    IsNotNullChecks wheter the existing value is not null
    IsNullChecks wheter the existing value is null
    LeqThe existing value is lower equal x
    LtThe existing value is lower than x
    Source: Query schema in CAML | Microsoft Docs

    In order to filter by query paramter, you have to define a filter query, depending on your datatype (string, integer, boolean..) you have to choose a different query value type.

    Logical Joins

    OperatorMeaning
    AndBoth query operations have to be fulfilled
    OrOnly one query operation have to be fulfilled
    Source: Query schema in CAML | Microsoft Docs

    Query Examples

    My blog would not keep it’s promise, If you would not find examples, which give you a fast way to adapt the scripts, so here we go!

    In my example, I am using my demo opportunities list. I have marked the names of the columns, the value types, the operands and the actual values bold. Mostly I am using the logical operator “eq”, but I think if you got the basic concept of this, you can adapt it to your solution easily and if not, we will find a way together.

    Example for boolean

    If you want to find items with TRUE values, you have to enter 1. For FALSE values, you have to make use of 0.

    If boolean should be true:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='Win'/><Value Type='Boolean'>1</Value></Eq></Where></Query></View>"

    If boolean should be false:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='Win'/><Value Type='Boolean'>0</Value></Eq></Where></Query></View>" 

    Example for choice

    If you want to filter for values choice values, you have to make use of a query like this:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='Product'/><Value Type='Choice'>SAP</Value></Eq></Where></Query></View>"

    Example for currency

    You have to enter the value of the amount without the currency sign.

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='DealSize'/><Value Type='Currency'>40000</Value></Eq></Where></Query></View>"

    Example for DateTime

    You have to format date times according to this format (ISO8601).

    yyyy-MM-ddTHH:mm:ssZ

    You can do this by appending -Format s, when creating the variable

    $CreationDate = Get-Date "16.06.2021 20:04" -Format s

    If DateTime should exactly match a specific date

    $CreationDate = Get-Date "16.06.2021 20:04" -Format s
    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='Created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>$CreationDate</Value></Eq></Where></Query></View>"

    If DateTime should be after a specific date

    Example: I want to find all items, created after 15.06.2021.

    $CreationDate = Get-Date "15.06.2021" -Format s
    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Gt><FieldRef Name='Created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>$CreationDate</Value></Gt></Where></Query></View>"

    If DateTime should be before a specific date

    Example: I want to find all items, created before 15.06.2021.

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Lt><FieldRef Name='Created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>$CreationDate</Value></Lt></Where></Query></View>"

    Example for GUID

    [GUID]$UniqueID= "b4ae9e9f-7103-459a-acb2-73573d035b36"
    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='UniqueId'/><Value Type='GUID'>$UniqueID</Value></Eq></Where></Query></View>"

    Example for integer

    In this case I want to find all opportunites with 2 stakeholders.

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='Stakeholder'/><Value Type='Integer'>2</Value></Eq></Where></Query></View>"

    Example for lookup

    Get-PnPListItem -List "Opportunities" -Query "<Query><Where><Eq><FieldRef Name='Contact'/><Value Type='Lookup'>Sus Spicious</Value></Eq></Where></Query>"

    Example for Note aka multi line text

    Get-PnPListItem -List "Opportunities" -Query "<Query><Where><Eq><FieldRef Name='Notes'/><Value Type='Note'>He was really curious.</Value></Eq></Where></Query>"

    Example for text aka string

    In this Query, I am looking for items, where the title equals Opp 3.

    Get-PnPListItem -List "Opportunities" -Query "<Query><Where><Eq><FieldRef Name='Title'/><Value Type='Text'>Opp 3</Value></Eq></Where></Query>"

    Example for user

    In this query, I am looking for items, where the authors UPN is Serkar@devmodernworkplace.onmicrosoft.com.

    Get-PnPListItem -List "Opportunities" -Query "<Query><Where><Eq><FieldRef Name='Author' /><Value Type='User'>Serkar@devmodernworkplace.onmicrosoft.com</Value></Eq></Where></Query>"

    Example for OR

    In this query, I am looking for items, where the value for Stakeholder is 1 or the value Win is yes.

    Get-PnPListItem -List $ListName -Query "<View><Query><Where><Or><Eq><FieldRef Name='Stakeholder'/><Value Type='Integer'>1</Value></Eq><Eq><FieldRef Name='Win'/><Value Type='Boolean'>1</Value></Eq></Or></Where></Query></View>"

    Example for AND

    In this query, I am looking for items, where the value for Stakeholder is 1 and the value Win is yes.

    Get-PnPListItem -List $ListName -Query "<View><Query><Where><And><Eq><FieldRef Name='Stakeholder'/><Value Type='Integer'>1</Value></Eq><Eq><FieldRef Name='Win'/><Value Type='Boolean'>1</Value></Eq></And></Where></Query></View>"

    Complete example

    $Url = "https://devmodernworkplace.sharepoint.com/sites/Sales" 
    $ListName = "Opportunities"
    
    Connect-PnPOnline -Url $Url -Interactive
    
    $AmountOfStakeholders = 2
    $ColumName = "Stakeholder"
    
    Get-PnPListItem -List $ListName -Query "<View><Query><Where><Eq><FieldRef Name='$ColumName'/><Value Type='Integer'>$AmountOfStakeholders</Value></Eq></Where></Query></View>"

    Troubleshooting

    I am getting to many items

    Error

    You get nearly every item in the list, but you are filtering for specific SharePoint items

    Cause

    Maybe you forgot the <View> part?

    Resolution

    Without view:

    With view:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Eq><FieldRef Name='Stakeholder'/><Value Type='Integer'>2</Value></Eq></Where></Query></View>"

    Exception from HRESULT: 0x80131904

    Error message:

    Get-PnPListItem : Exception from HRESULT: 0x80131904
    In Zeile:1 Zeichen:1
    + Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><gt ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (:) [Get-PnPListItem], ServerException
        + FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Lists.GetListItem

    Cause:

    You did not care about the case sensitivity of the logical operands

    Resolution:

    Wrong:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><gt><FieldRef Name='Created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>$CreationDate</Value></gt></Where></Query></View>"

    Right:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Gt><FieldRef Name='Created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>$CreationDate</Value></Gt></Where></Query></View>"

    Field types are not installed properly

    Error message in german

    Get-PnPListItem : Mindestens ein Feld ist nicht richtig installiert. Wechseln Sie zur Listeneinstellungsseite, um diese Felder zu löschen.
    In Zeile:1 Zeichen:1
    + Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Gt ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (:) [Get-PnPListItem], ServerException
        + FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Lists.GetListItem

    Error message in english

    Get-PnPListItem : One or more field types are not installed properly. Go to the list settings page to delete these fields.
    In Zeile:1 Zeichen:1
    + Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Gt ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (:) [Get-PnPListItem], ServerException
        + FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Lists.GetListItem

    Cause

    You did not care of the case sensitivity of the column name

    Resolution

    Wrong:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Gt><FieldRef Name='created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>$CreationDate</Value></Gt></Where></Query></View>"

    Right:

    Get-PnPListItem -List "Opportunities" -Query "<View><Query><Where><Gt><FieldRef Name='Created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>$CreationDate</Value></Gt></Where></Query></View>"

    Bild von Deedee86 auf Pixabay