This blog post explains how to remove unwanted characters from a string in a Power Automate flow. These can be regular characters or unprintable characters, such as control codes.
My example string:
<note> <to>[email protected]</to> <from>Michael Knight</from> <heading>Reminder</heading> <body>Don't forget to clean KITT this weekend!!! </body> </note>
The aim is to remove the apostrophes, exclamation marks and the two non-printable characters. The non-printable characters are visible in Notepad++ :
As you can see from the screenshot, the non-printable characters are the SOH (Start of heading) and BEL (Bell).
It is possible to replace all unwanted characters without any apply to each loops for maximum speed. If you just want implement this flow without understanding how it works, please just skip to the section Quick Paste Scope Code.
Table of contents
Flow Overview
Here is an image of the entire process:
Step by Step Explanation
Here is a step by step explanation of how the flow works.
Create a compose action to define the characters to remove
In the first step, a compose action is defined which contains a JSON array of characters to be removed from the string. If for example you wanted to remove the characters A, B and C, you would define it as:
[ "A", "B", "C" ]
Create a compose action to define the string to have characters removed.
This step is very simple, simply put the string you want to remove characters from in here (you could also use dynamic string content)
Create a Select action to remove any invalid characters from the string.
Most of the work is done within this select action. This is much faster than using an apply to each method as the execution time is near instant. But it does require some explanation. In the from field use the expression:
range(0, length(outputs('String_To_Modify')) )
The range function is useful for all sorts of things, but what we are using it for is to create an array of numbers. Each number represents one character of the string. If the string to have characters removed was:
I love Power Automate!
The range expression would produce the following array:
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21 ]
The array contains an entry to represent each character in the string.
In the map section of the select action the expression used is:
if
(
contains(outputs('Unwanted_Characters'), substring(outputs('String_To_Modify'), item(), 1)),
'',
substring(outputs('String_To_Modify'), item(), 1)
)
This if expression checks through each character of the string and checks to see if that character is contained within the array of characters to be removed. If it is not in the array to be removed, then it is simply output as is, if it is found, then an empty string is output instead.
Reconstruct the String
The Select action expects it’s input to be an array, which we have provided using the range expression. It also outputs an array. If I wanted to remove the exclamation marks I would specify the Unwanted Characters compose action as:
[ "!" ]
And the select action would produce the following output:
[ "I", " ", "l", "o", "v", "e", " ", "P", "o", "w", "e", "r", " ", "A", "u", "t", "o", "m", "a", "t", "e", "" ]
Note how the final character which was an exclamation mark has become an empty string.
Reconstruct the string
Now that we have an array of characters with all of the unwanted characters removed. We can rebuild the string with a simply compose action containing the expression:
join(body('RemoveUnwanted'), '')
This will convert the array back into a string which we can then use in further actions within the Flow.
Brief overview of how the flow works:
- A variable called string is defined which contains the complete string including the invalid characters.
- A compose action called RemovalChars is created, which is an array of characters that you’d like to be removed.
- An apply to each loop is created using the array RemovalChars as it’s basis.
- Inside the loop, a compose action called “removeCharacter” uses the replace function to remove the character in the current iteration of the loop.
replace(variables('string'), item(), '') - The variable string is updated with the output from the compose action.
- Inside the loop, a compose action called “removeCharacter” uses the replace function to remove the character in the current iteration of the loop.
- Finally, the content of the string with the characters removed is placed inside a compose action, so that you can check if the output is as expected.
Non-printable characters
As I mentioned in the start of the post, it is also possible to remove special characters, that are non-printable. In my example I wanted to remove the SOH (Start of heading) and BEL. I represented these as:
[ "u0001", "u0007" ]
These numbers represent the Unicode values of these characters. Which you can lookup on unicode-table.com.
Quick Paste Scope Code
If you want to implement the above in one of your flows, then you can copy and paste the scope above into your flow. Simply do the following:
- Copy the code block below into your clipboard.
- Go into the flow editor and add a new action.
- Click on My Clipboard.
- Press CTRL-V.
- Click on the newly Visible action Remove Unwanted Characters.
- Edit the characters and string as per your requirements.
{
"id": "09aad5c1-1777-4c50-91b6-0698-3fc31b57",
"brandColor": "#8C3900",
"connectionReferences": {},
"connectorDisplayName": "Control",
"icon": "data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMzIiIGhlaWdodD0iMzIiIHZlcnNpb249IjEuMSIgdmlld0JveD0iMCAwIDMyIDMyIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPg0KIDxwYXRoIGQ9Im0wIDBoMzJ2MzJoLTMyeiIgZmlsbD0iIzhDMzkwMCIvPg0KIDxwYXRoIGQ9Im04IDEwaDE2djEyaC0xNnptMTUgMTF2LTEwaC0xNHYxMHptLTItOHY2aC0xMHYtNnptLTEgNXYtNGgtOHY0eiIgZmlsbD0iI2ZmZiIvPg0KPC9zdmc+DQo=",
"isTrigger": false,
"operationName": "Remove_Unwanted_Characters",
"operationDefinition": {
"type": "Scope",
"actions": {
"Unwanted_Characters": {
"type": "Compose",
"inputs": ["'", "!", "@", "£", "^", "=", ",", "%", "[", "]", ";", "~", "?", "<", ">", "|", "u0001", "u0007"],
"runAfter": {},
"description": "List of unwanted characters to remove (Compose Action)"
},
"String_To_Modify": {
"type": "Compose",
"inputs": "I love Power Automate!",
"runAfter": {
"Unwanted_Characters": ["Succeeded"]
},
"description": "String from which you want to remove characters (Compose Action)"
},
"RemoveUnwanted": {
"type": "Select",
"inputs": {
"from": "@range(0, length(outputs('String_To_Modify')) )",
"select": "@ifrn(rn contains(outputs('Unwanted_Characters'), substring(outputs('String_To_Modify'), item(), 1)), rn '', rn substring(outputs('String_To_Modify'), item(), 1)rn) "
},
"runAfter": {
"String_To_Modify": ["Succeeded"]
},
"description": "Select action to remove unwanted characters"
},
"Re-join_Original_String": {
"type": "Compose",
"inputs": "@join(body('RemoveUnwanted'), '')",
"runAfter": {
"RemoveUnwanted": ["Succeeded"]
},
"description": "Reconstruct the string"
}
},
"runAfter": {},
"description": "http://tachytelic.net/2020/10/remove-unwanted-characters-power-automate-flow/ "
}
}
This is a really fast and efficient solution for removing characters from a string in Power Automate. I Hope it helps you out, and if it does, let me know in the comments!
Adam W says
I would like to keep the list of special characters in a SharePoint list so that users can easily update them and then pull the special characters from the list into the flow. How can this be done?
Tristan Holt says
In the “Replace Unwanted Character” compose action, what is ‘NewFileContent’?
Mike M. says
Thank you, Paul. Method 2 is exactly what I needed!
mijo says
Hello, thanks for your job. I have an error on “substring(outputs(‘String_To_Modify’)” because it cannot put array for substring. It needs string. Any ideas ?
Julie Sebby says
It appears the Select action has changed. Map now has an entry for “Key” and “Value”. How would this be updated?
Paulie says
Hi Julie,
Sorry, the blog post does not make it clear. The select action must be put into text mode. As per this image:
Hope this helps.
Paul
BB says
HI and thanks for this great explonation.
I can se if “string to modify” has spaces then it removes them to, how can i prevent it to remove spaces. Ex. Department USA – then my final compose is DepartmentUSA without space.
Paulie says
It should retain the spaces, do you perhaps have a space character in your array of Unwanted Characters?
Amanda Johnson says
Since I can copy/paste the code, but not use it (it doesn’t do anything when I click in clipboard), I am reconstructing the flow manually. However, I’m confused — The scope shows 4 compose actions, however in the “Brief Overview” section you reference a compose action called “RemovalChars” and another called “removeCharacter”, as well as the loop to perform the action. Do you have those documented as well?
J Herschel says
I have the same copy/paste issue, something to do with the RemoveUnwanted step, I think in the select statement. I had to remove that step before copying and pasting…
Filip Berndtsson says
Hi, Thank you very much for a thorough walkthrough. When I copy the code block above and paste it into the clipboard i get an error message that sais “Could not get any expression token of type LeftParenthesis”. Do you know how to solve this?
dJIA says
150561 148357 very nice post, i definitely adore this web site, keep on it 9876
Dan says
Hi Paulie, is it possible to have a flow that works kind of in reverse, i.e. list valid characters and remove anything that is not in that list (I don’t know all the invalid characters I may get in my input string but I do know that I’m only interested in keeping the letters a-z)
Wade Murdock says
The “Map” field’s “if” statement is giving me “Enter a valid json.” error. Very annoyingly vague. I both copy and pasted, and manually typed it out.
wad11656 says
Copy-and-pasting your entire Scope doesn’t work either: I keep clicking on the “Remove Unwanted Characters” from “My clipboard” to paste it in, but nothing happens. Maybe there’s been updates that make the code incompatible. (Maybe the “Map” code)
Joe Faurote says
I struggled with this too… you can’t seem to Quick Paste Scope Code anymore, however, you can reconstruct the steps manually. Make sure to use Expressions when cutting and pasting the logic for the Select composition for both the From
range(0, length(outputs(‘String_To_Modify’)) )
and Map (after selecting text per Paulie’s response)
ifrn(rn contains(outputs(‘Unwanted_Characters’), substring(outputs(‘String_To_Modify’), item(), 1)), rn ”, rn substring(outputs(‘String_To_Modify’), item(), 1)rn)
This gets tricky quickly.
Mark says
Same issue as BB i.e. I DO NOT have a space character in my array of Unwanted Characters BUT the Select step is removing them anyway.
Shoeb Memon says
Same issue as BB and Mark. DO NOT have a space character in the array of Special Characters but the Select step removes them.
Brian says
Make sure to turn off experimental features in the editor. This messes with the behaviour when editing the Select function. It also affects the output.
If you want to remove backslash “\” use “\\” otherwise spaces are removed.
If you cannot paste the code block and get a left parenthesis error: Replace all @ with a @@
Ryan says
Brian, I used the “\\” instead of “\” and the spaces are still being removed. I also tried changing the forward slashes “/” with “//”, as well as “|” with “||” which did not help. Are there any other characters that would cause this?
Here is my current array: [ “‘”, “””, “#”, “%”, “*”, “:”, “”, “?”, “//”, “\\”, “||” ].
Also, I’m not familiar with turning off the experimental features, I can’t seem to find that setting.
Ryan says
I made a workaround, so it doesn’t remove spaces. It replaces the spaces with an underscore before removing the unwanted characters, then will replace the underscores with a space after removing the unwanted characters. This won’t work if the ‘_’ is listed as one of the unwanted characters, but you can add similar operations to work around that.
On the front end: replace(outputs(‘String_To_Modify’),’ ‘,’_’)
On the back end: replace(outputs(‘String_To_Modify’),’_’,’ ‘)
Brian says
Experimental features
In the editor click on gear wheel top right. Click on View all Power Automate Settings. Experimental features is an on|off button at the bottom of the dialogue.
I have this [ “‘”, “:”, “@”, “£”, “^”, “=”, “,”, “%”, “[“, “]”, “?”, “”, “|”, “/”, “\\”, “*”] in unwanted characters. Spaces are not removed.
I modified the Select statement so that
From: Range statement
Map: Letter | if statement
added another Select in text mode – This outputs an array from the JSON.
From: output from previous Select
Map: item()[‘Letter’]
Replaced the Compose with Join expression to Join function with comma (or any of the ‘removed chars’) as the separator
Composed with replace expression to remove the commas
Brian says
I believe “\\” works because \ is the escape character. If there is a problem with “|” try “\|”
Wim says
Pretty straightforward to get this to only include a list of characters, just swap the 2nd and 3rd parameters in the Map expression in the Select action, if I define the inclusion array in a Compose called ‘include’ that expression is
if(contains(outputs(‘include’),
substring(triggerBody()[‘text’], item(), 1)),
substring(triggerBody()[‘text’], item(), 1),
”)
The rest of the flow is just the same join action – though I named my Select action ‘include_chars’,
join(body(‘include_chars’), ”)
The main gotcha, as Paulie says, is that you must have Select in text mode, where you don’t see the T icon.
Sai says
Thanks!
morish says
Need to remove all the rn
@ifrn(rn contains(outputs(‘Unwanted_Characters’), substring(outputs(‘String_To_Modify’), item(), 1)), rn ”, rn substring(outputs(‘String_To_Modify’), item(), 1)rn) ”
from the above condition and also you need to select dynamic condition and then enter the condition
Venugopal Mallavajjala says
Thank you so much for this solution, it worked like a charm. It saved my time!
Ralf says
This solution is very good. Thank you so much for sharing. I had some difficulties to get it work. The following is a summary of tips from older comments, that helped me getting it to work.
To make copy and paste into Power Automate work, I had to replace all @ with @@ in plain text editor.
After copy and paste into Power Automate I had to change the following in Power Automate Editor:
1. Remove the [ and ] in Step “Unwanted Characters” text box.
2. In Step “Remove Unwanted” I had to remove the @ and remove all the rn then select the text, cut it and add it into the flyout dialogue to specify dynamik expressions. For both “from” and “card” input boxes.
3. In “Re-join Original String I had to do the same as explained in Step 2.
Hope this helps others to get a working solution faster.
MadPrince says
After removing unwanted characters, how do you continue after joining ? I want to make it a csv file again to be able to input it to the database.
Niels says
@Ryan your spaces are removed because of “””. If you replace this by “\””, spaces will be kept. At least this worked for me.
Scoutriley says
I have a list of unwanted characters, yet I noticed that the u’s, 0’s, 1’s & 7’s were being deleted when I included [ …”u0001″, “u0007″ ] at the end. I tried the following variations as well, [ …”\u0001”, “\u0007” ], [ …u0001, u0007 ], and [ … \u0001, \u0007 ]. None seemed to return any other result. Does anybody know how to get Unicode characters to work in the omission list? I appreciate any help you can provide.
Steve Morley says
What about the enter key? I have a flow that fails when someone presses enter in a multi line text field.
I have it working for all the other special characters.
MrAce says
Hey, I think I know why spaces, commas, double quotation marks, and new lines are removed… I think that because the “Unwanted Characters” compose action contains all of these characters in the formatting of the list.. the code sees them and removes them! So I took out all of the formatting so my compose action for “Unwanted Characters” just looks like
{}[]`~\|
all by itself, no formatting. This solved my issue.
Thanks for this tutorial by the way! It really helped me with exactly what I was looking for! If, by your testing, removing the formatting from the “Unwanted Characters” compose action works just as well without the bugs people have described, I think it would be helpful if you updated the tutorial so the bug is “patched.” 🙂