Web Development

Ask something
Discussion Locked

Special Character Remover

0 helpful
12 replies

by coopcaborojo 1 view

Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

Hello!

I'm working on a migration from Cobol (85) to Oracle. I convert the Cobol data to ascii but Im having problem with special characters on the original data. I was thinking about developing a spec.char.remover routine on Cobol but, i know is gonna be a pain... Does anyone knows a "quick and dirty" way to remove special characters from a text file?

Carlos

12 replies

by coopcaborojo

I d/l the program but it doesn't work... When I double click on it it appears a black window and then dissapear...

by coopcaborojo

Sometimes during the data entry users tend to press keys like /f1/ or /page up/ which are considered by the accept command as "thrash data"(or as special characters)and the applications stores it in the X fields...

by klaasvanbe

Are those characters stored as ASCII or another character set e.g. EBCDIC?
In other words: What's the platform?

by luckyleatherneck

First define the full text area, like PIC X(132),and redefine it as PIC X(001) OCCURS 132 TIMES.

Then in your procedure, move your data into the full text field.

Now, within a perform index through the single byte occurrances doing the IF NUMERIC and IF ALPHABETIC checks to remove anything that's neither numeric (F0 - F9) or alphabetic (C1 - E9). All that is left is the good text, unless you want to also remove the plusses, minuses, etc., which you can easily look for with IF '+' and so on.

If you are using SPF as the editor for your program, there are ways to have it search your program or text document for non-display characters and automatically replace them with whatever you want -- usually a space.

By the way, EBCDIC to ASCII conversion is only good for display data -- not packed.

let me know if this helps.

Lucky

by johnnymegabyte

In Oracle you can create a series of variables, each with the assigned HEX of the special characters.
Example: TAB = chr(9)
X_TAB char(1) := chr(9);

Then, use a REPLACE statement on your input data string, to remove the special characters to a blank or null.

You may want to use an INSTR command to check if a special character exists first, then perform the REPLACE in a LOOP an work character-by-character.

That's how I would do it. And I'm sure there are more ways.

by justin james

Any language can do this, but using Perl will make it go super fast. Just use a readline() loop to get the file line by line, use a regex to retain only the characters you want:

$line =~ s/[^characters to keep in here]//mxs;

Then print $line out to a new file, or to the screen and redirect the output.

J.Ja

by johnnysacks

Perl can be a pain but it shines for this kind of Sh#t work (cleaning up other peoples messes)

All on one command line:
perl -p -ibak -e"s///g"

You're left with original renamed with bak appended and modified file same name as input file

by justin james

Yes, Perl can be a headache, especially when someone slapped together a ton of code that is barely held together by implicit variables...

If one didn't like Perl, sed could do this just as easily. :)

J.Ja

by richard_simcoe

13 is the ascii decimal value for carriage control and 10 is the ascii decimal value for line feed. The chr(n) function returns the character value. The || is the concatenation of the two. Use the replace function
-- replace (remarks,chr(13)||chr(10),' ')
An example follows:

SQL> select remarks from
2 iws_well_remark where uwi = '100013603906W500';

REMARKS
--------------------------------------------------------------------------------
PENN WEST DISPOSITION
EFFECTIVE DATE: OCTOBER 01, 1998
CONTRACT FILE: 05447

1 select replace (remarks,chr(13)||chr(10),' ') from
2* iws_well_remark where uwi = '100013603906W500'

REPLACE(REMARKS,CHR(13)||CHR(10),'')
--------------------------------------------------------------------------------
PENN WEST DISPOSITION EFFECTIVE DATE: OCTOBER 01, 1998 CONTRACT FILE: 05447

This discussion is closed to new replies.