The forum will be down for maintenance at some point from Friday, November 16 at 19:00 EDT until Sunday, November 18 at 23:59 EDT. Downtime will be minimized but the exact timing is unknown.

Note: This question is about Sybase Adaptive Server Enterprise 15.x or 16. I'm not fully sure, if this forum is the correct one. If you know a better one, please be nice and direct me to it. Thanks.

We store text data in UTF-8 in Sybase VARCHAR columns with an UNIQUE INDEX on this column. We face a dup key violation when inserting the following two Arabic words:

بمشًْذالسيّد
بمشًذالسيد

A deeper analyzing of the code points show in hex:

d8  a8
d9  85
d8  b4
d9  8b
d9  92  <---- ARABIC SUKUN  
d8  b0
d8  a7
d9  84
d8  b3
d9  8a
d9  91  <---- ARABIC SHADDA 
d8  af

d8  a8
d9  85
d8  b4
d9  8b
d8  b0
d8  a7
d9  84
d8  b3
d9  8a
d8  af

i.e. the two words are "nearly" identical and the first word only contain two additional chars ARABIC SUKUN and ARABIC SHADDA which have some meaning in pronunciation of the words (I'm German and have no further knowledge about Arabic, I don't even know what the above words mean. They came in as biographic data into our database.).

Is it possible that Sybase ASE throws away the two codepoints 0xd991 and 0xd992 when creating the INDEX as some kind of normalization? And if so, is this somehow something which could be avoided by configuration.

asked 23 Jul, 04:15

gurucubano's gravatar image

gurucubano
11112
accept rate: 0%

2

This forum is about SQL Anyhwere. For ASE, you might use the forum in the SAP Community here.

(23 Jul, 04:32) Volker Barth

Agree with Volker. This is a wrong forum :(

But to me it sounds that the issue might be related to the DB collation. I googled a little, but didn't find this concept for ASE :( Thus I don't know either.

(28 Jul, 16:51) Vlad
Be the first one to answer this question!
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×52
×16
×2

question asked: 23 Jul, 04:15

question was seen: 1,680 times

last updated: 28 Jul, 16:51