Professional Documents
Culture Documents
Abap Material by SP Rao PDF
Abap Material by SP Rao PDF
Share 1 0 Tweet
It’s very well known that the most errors during data load occur due to invalid characters in data inflow.
The previous my blog was dedicated to invalid characters in the texts of infoobjects. Hereinafter, all is to be said is
applied to invalid characters in characteristic values only. Though, provided here information is valid for non-unicode
systems, I believe that the big part of it might be applied to Unicode systems also.
It’s amazing that even some very experienced people share some delusions about invalid characters. So, here is a list of
myths related to invalid characters in characteristic values.
Myth 3: The SPACE symbol is a special one and has to be added to allowed characters in RSKC (if we
want to have the SPACE in characteristic values)
Not all characters are allowed in characteristic values of infoobjects. There is a predefined set of allowed characters. All
notes, HowTos and help documents state that this set include the following characters (see, for example, the OSS Note
#173241 – “Allowed characters in the BW System”):
!"%&'()*+,-/:;<=>?_0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ
As you can see, there is no SPACE symbol. And I remember that there were even threads in BI forums related to
inserting the SPACE in RSKC by pressing ALT-0160 or ALT-255.
Actually, the default set of allowed characters DOES INCLUDE the SPACE symbol. You can see it by yourself:
E_DEFAULT_CHAR = G_C_ALLOWED_CHAR
There you can see the default allowed chars, and the SPACE is the first one:
!https://weblogs.sdn.sap.com/weblogs/images/32317/w01.JPG|height=154|alt=image|width=567|src=https:
//weblogs.sdn.sap.com/weblogs/images/32317/w01.JPG|border=0!
There is also a list of clearly forbidden characters: from hex code 00 to 1F.
Additionally, you may just make a little experiment - typing in some characters with spaces in characteristic value of
infoobject of type CHAR:
You’ll see that SPACE symbol is allowed without any special settings in RSKC.
1 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
Since invalid characters are often represented in BW as # sign, many beginners believe in this myth.
It’s a delusion. The system uses # character for all hexadecimal values, for which a code page does not know the
appropriate character to use. This is the case for invalid characters, and hence, this sign may represent ANY invalid
characters in BW messages.
The presence of this sign in RSKC will not prevent failure of data load with invalid characters. It only will allow presenting
the # sign itself (hexadecimal code 23) if it is not alone in the value.
Myth 5: ALL_CAPITAL setting in RSKC along with the other special characters solves problems with
invalid characters
The example:
It’s wrong. Put NOTHING else than "ALL_CAPITAL" in transaction RSKC, otherwise the system won’t recognize it. Only
additional special characters set will be recognized. ALL_CAPITAL will be treated simply as letters A, L, _, C, P, I and T to
be allowed. Though, these symbols are already permitted by default.
As you can see in the picture to Myth 3, characters with hex codes from 00 to 1F are forbidden. These very characters
cause the most of problems during data load.
Remember that pressing the keys like TAB, ENTER, arrows, BACKSPACE creates exactly these not permitted codes.
Hence, some validation of incoming to BW data must be made if data is created in an application where a user types in
information and the application doesn’t check it.
By calling some ABAP functional modules returning l_userdef_char (user defined allowed characters) you’ll not get a set
of permitted symbols in case of ALL_CAPITAL setting. You’ll simply get this very ‘ALL_CAPITAL’ string.
‘ALL_CAPITAL’ is interpreted by the other modules that check languages installed in the system.
ALL_CAPITAL_option, as it is said in the How To "Permitted Characters in Characteristic Values” allows you to use
all the characters that are capitals in the local language (the language in which the batch processes run). The result is
that the loading process becomes language dependent.”
There is a common belief that with this option only uppercase letters are allowed. That’s not true! If you check
‘Lowercase Letters’ for an infoobject, you can load data with lowercase letters too! Even for foreign languages installed
in the system.
2. Since except ‘ALL_CAPITAL’ string you cannot set another characters in RSKC, how to allow the
other special characters like #, ~ etc. (not hex 00-1F codes)?
You do not need this. ALL_CAPITAL option takes care about such symbols too!
So, as we see, ALL_CAPITAL is very powerful option and allows the maximum number of signs in BW.
It cannot manage only a small set of exceptions. For completeness of information it makes sense to provide here a
quotation from the mentioned How to:
The following characteristic values are not permitted and lead to errors in the system:
Values that consist only of the character #. This is because blank entries in variables are marked with # (no entry =
space).
Values that begin with the character !. The system deletes these values.
Control characters with the hex-display 00 to 1F (valid as of BW 3.0A if small letters were previously allowed).
Characters are not allowed if they are represented by a hex code that has a small letter in one of the installed
languages. You could, for example, only allow the capital Ö if no language is installed where the corresponding hex
code is a small letter. In this example, it would not be possible to have Russian installed, because the hex code for
capital Ö means a small sch in this example.
The symbols for the US Dollar ($), British Pound (£), Japanese Yen (¥) and the Euro are not allowed by default.
The symbol for the US Dollar ($) is allowed in BW customizing. The other special currency indicators lead to errors in the
system.
In the InfoObject maintenance screens or in the transfer rules, you need to create a transfer routine for any currency
fields and other characteristic values that have special currency indicators. This transfer routine converts the invalid
characters into valid characters or character sets. This conversion is mandatory for currency fields. The currency codes,
into which you need to change the currencies, are stored in BW customizing under General Settings -> Currencies ->
Check Currency Codes (or ISO4217). The key must agree with the keys in the currency tables.
2 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
Most of the special currency indicators can be assigned to three-character currency codes. For example, the $ dollar
symbol is converted into USD for US dollars or AUD for Australian dollars.
As we can see, the most errors during data load belong to the first three categories provided in the quotation.
How to deal with these invalid characters? The best option is to write an ABAP code and place it into a routine in transfer
rules. Here, in the forum I saw many examples of such a code.
Though, I don’t agree with the statement that “for characteristics to be used for navigation – lower case is not permitted”.
Queries showed results without problems. Drill down by navigational attribute also worked fine. I had ‘ALL_CAPITAL’
setting in RSKC.
I think that the main prerequisite for successful data load is a load of master data before transactional ones.
So, in case of loading data in infoobjects with lowercase letters flag checked, there is no necessity to translate
everything to upper case.
Unfortunately, as far as I remember, all the code examples for removing invalid characters posted in the forums deal with
any settings in RSKC, EXCEPT ALL_CAPITAL.
The code below deals with any settings in RSKC. It doesn’t compare incoming text with permitted characters. It simply
replaces by space the following invalid characters:
The code assumes that the field for which this routine is created is of CHAR type. Moreover, if lowercase letters may
come from the source system, flag ‘Lowercase Letters’ should be checked for the infoobject.
RESULT = TRAN_STRUCTURE-ZZZ.
3 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
Hopefully, some ABAPers will come up with the solution for Unicode systems also.
Resume
If master data SIDs were created successfully, then problems with ODS activation and loads into cubes may arise
due to:Lowercase letters. Solution: check the ‘Lowercase Letters’ for appropriate infoobject or apply
TRANSLATE RESULT TO UPPER CASE.
statement in a routine in transfer rules (it might be done in a formula too) if you don’t care that all texts will be in
upper case only.
Forbidden characters listed earlier. Solution: apply a routine either posted in forums or provided here.
(2 ratings)
Share 1 0 Tweet
30 Comments
Something you really need in any system. Here is a completely compatible variant that also checks if
the InfoObject allows lower case letters.
Simply leave all the logic to SAP, don't care about Unicode, etc. The only thing that is (a bit) wrong is
that my code eliminates inner # even if it would pass otherwise. Not a big thing to add.
Best regards
Dirk
Like (0)
Hi Dirk,
4 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
Best regards,
Eugene
Like (0)
Superb ... but ... why to have the call function in the DO ... ENDDO ? Not needed I
guess.
Like (0)
namuduri VISHALAKSHMI Jun 15, 2007 2:35 AM (in response to Nicolas Van Mello)
Hi,
ST-FÉLICIEN-QC-CA
QUÉBEC-QC-CA
even i have tried RSKC but no use any conversion roune should be written
for these symbols usually we get it in insert symbol in word so ..........pls
send me a mail visalakshi.sapbw@gmail.com
thanks in advance
vishali...
Like (0)
Hi Eugene
I am new to BI..but still I am working in BI..Could you pls help me with regard to the
Invalid character..??
Before that I want to know ...NORMALY
1)SOURCE FIELD LEN CAN BE DIFFERENT FRM OUR TARGET FIELD LEN
2)i have an invalid character error while I am activating the Request..
So if I use ur code..is that I can resolve for the following error..
Error: Value 'Not Applicable' (hex. '4E6F74204170706C696361626C65') of
characteristic ZANS_LAB contains invalid characters
Error: Value 'Eda Lam' (hex. '456461204C616D') of characteristic ZPENAME
contains invalid characters
Could u pls revert me ASAP..
Thanx in advance
Like (0)
BI Learner BI Learner Nov 30, 2007 1:10 AM (in response to Dirk Herzog)
Hi Dirk
I am new to BI..but still I am working in BI..Could you pls help me with regard to the Invalid
character..??
Before that I want to know ...NORMALY
1)SOURCE FIELD LEN CAN BE DIFFERENT FRM OUR TARGET FIELD LEN
2)i have an invalid character error while I am activating the Request..
So if I use ur code..is that I can resolve for the following error..
Error: Value 'Not Applicable' (hex. '4E6F74204170706C696361626C65') of characteristic
ZANS_LAB contains invalid characters
Error: Value 'Eda Lam' (hex. '456461204C616D') of characteristic ZPENAME contains
invalid characters
Could u pls revert me ASAP..
Thanx in advance
Like (0)
Iris Siebrands Jul 18, 2008 9:05 AM (in response to Dirk Herzog)
Hi Dirk,
thanks for that really comfortable solution! Since I've been a bit busy with simply trying to find
this essential peace of information - 2 years after it had been posted - I'd very much
appreciate if SAP could include this central and basic information e.g. in the standard online
SAP-library - for example where the general settings for characteristics are described.
Thanks to all of you for your troubles to help the SAP developer community ... ;-)
Iris
5 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
Like (0)
Hi Eugene,
I have been using this setting in RSKC since November 2000 in all BW installations, but it is good to
know that all my recommendations of using it were true. I now know more about it and can really justify
it's inclusion in RSKC.
Thanks
Ian
Like (0)
Hi Ian,
The most of information provided in the blog is of experimental/experience nature (in the docs
and help you may find just some peaces of truth). I don't exclude that some of the statements
might wrong in some particular circumstances.
So, I welcome any feedback on the matters (either confirming or the opposite).
Best regards,
Eugene
Like (0)
I've tried to use Dirk's code in a transfer rule and had to make a small correction so I thought I'd let you
know.
Instead of entering directly the infoobject's name to the function module I had to declare a 30 character
string, assign it with the infoobject's name and use that string when calling the function.
Like (0)
Krishna Joshi Dec 12, 2006 8:28 AM (in response to Pedro Pais)
Hi Pedro,
Can you please send me working code at nutkhat@optonline.com ? I declared char string
and assinged to infoobject but still having some problems.
Kind Regards,
Krishna
Like (0)
Krishna Joshi Dec 12, 2006 10:00 AM (in response to Pedro Pais)
Hi Pedro,
Krishna
Like (0)
Very good presentation about the argument. In the past I spent too much time to found ALL_CAPITAL.
Regards,
Sergio
Like (0)
Like (0)
6 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
ABORT = 0.
Like (0)
It's really very helpful. Now we have a code working in unicode system also. Thank you very
much Xiuhai Yue!
Like (0)
Like (0)
Meher Dheram Nov 13, 2008 10:49 AM (in response to Xiuhai Yue)
Hi,
I am having an issue with invalid characters. We are on BI 7.0, Unicode. Could you
e-mail me the code at mehervinay_7@yahoo.com.
Thanks,
Meher
Like (0)
Like (0)
I tried the above code to remove invalid data and it did not work.
Like (0)
Shouldn't it be included in some control on R3 system - that user can not enter invalid characters in
texts? It will solve 90% of problems like that!
Like (0)
Unfortunately a lot of SAP BI/BI consultants are only concerned with fixing the problem at the
surface level, instead of addressing it at the root cause.
Invalid characters are usually caused by bad entry of data in the source system (ECC, or
non-SAP sources). These must be fixed in the source system, instead of patching the
character values in BI, because:
1) Data mismatch: You won't be able to match up the values between BW and the source
systems, if you start patching up the illegal characters.
(You might as well be transforming every single $100-sales order into $1,000 sales order to
mark up your reported earnings!)
3) Future support: If the data from the source system is loaded into multiple DSOs or cubes,
are you going to add the function module and call it in each and every transformation involving
the affected InfoObject/field?
7 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
#3 is usually the reason why most consultants are not bothered by this issue - because they
never stick around the client's company long enough to see things in the long-term (support)
perspective.
Like (0)
Yes, it's exactly what I wrote above: checking of invalid characters in the incoming
datastream in the source system should be dominant.
Like (0)
These days I just search for help on Blog and not in forums.Thanks Eugene ! Your blogs have been of
great use earlier as well ! :-)
Like (0)
Like (0)
Hi Eugene,
Thanks for your wonderful blog which cleared so many doubts. But i am still facing one issue even after
enablling ALL_CAPITAL in RSKC. In ECC some texts are maintained independent of languages as this
represent the name of a person(Data source 0RESPNO_TEXT). The text description includes special
characters like Ššžè..
These are not working in BW. I am getting # in the place all these characters.
Please help me ho to resolve this issue.
Like (0)
Eugene,
I have used the code given by Dirk and some local changes in my system.. however I have a unique
problem .. I have a file that has Chinese , Korean , Japanese etc and I am having problems in loading
this file. Do I have to :
2. Log into the system in Chinese or Japanese and load the file ( The file comes in to the app server
and I would eventually want to load the same through a process chain )
3. Is there anything I can do in terms of settings to be able to load this file while logged into the system
in good old English..?
Thanks ,
Arun Varadarajan
Like (0)
Eugene Khusainov Dec 15, 2010 11:35 AM (in response to Arun Varadarajan)
Sorry, Arun,
Like (0)
Arun Varadarajan Dec 15, 2010 1:08 PM (in response to Eugene Khusainov)
We are trying to see if installing the language packs will help. Once this is working , I
should be able to put together a nice blog on the same...
Like (0)
Arun Varadarajan Dec 16, 2010 7:24 AM (in response to Arun Varadarajan)
I set RSKC to ALL_CAPITALS and the data load went through without
issues... thanks for everything....
Like (0)
8 of 9 12-04-2014 12:41 PM
Invalid characters in SAP BW 3.x: Myths and Rea... | SCN http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/200...
9 of 9 12-04-2014 12:41 PM