[Padre-dev] Fwd: STC and unicode issue in Padre

[Gabor Szabo]

Another pair of messages.

---------- Forwarded message ----------
From: Gabor Szabo <szabgab at gmail.com>
Date: 2009/7/6
Subject: Re: STC and unicode issue in Padre
To: Carsten Zandecki <service at mediaminds.de>


Hi Carsten,

it's a pity we are not discussing this on a mailing list for others to
comment too.
Is it ok with you if I forward our discussion to the Padre mailing list ?


On Mon, Jul 6, 2009 at 10:39 AM, Carsten Zandecki<service at mediaminds.de> wrote:
> Hi Gabor,
> my operating system is WinXP Professional. I just can help you with
> (windows) knowledge ... Never really used linux/unix systems ... just the
> basics at work, cause I have my own business as webdesigner and programmer
> here in Germany. For that job i started in 2005 the development of my own
> text editor in wxPerl called "PED". It is just for my personal needs at work
> and also just a hobby ... a lot of undone parts in it ... it never has been
> released ... maybe i will release it someday ... But it works fine ...


> If you like just see screenshot ;)
> http://www.mediaminds.de/ped.jpg


wow, nice!

Even if you don't intend to package it or support it for others it
would be nice if you could
add the "Perl license", zip the source code and put it on your server
so we can learn
from it.

So we can steal ideas from you :-)

>
> If you have a windows system running i can send you an executable of my text
> editor just to see if it works with hungarian charsets. It would be
> interesting to me, too. If yes, i can send you my i/o perl code to ensure
> that this part is on the way in Padre.

I have one in a VirtualBox so I'd be happy if you sent me the executable.

> Once i installed Padre just to check it out. And it is really good, but
> often crashes on Windows. I'm a really bad programmer ;) and for sure never
> will be a good one ... but maybe I can give you some tips, which might make
> Padre more stable (at least on Windows). Which are the following:

Well, some issues have been resolved already since 0.27 but I am sure
there are still
many issues. If you try it again and encounter a problem please let us
know how to
reproduce it.

>
> 1. I don't think that it is a good idea to add a new scintilla instance for
> each file, cause especially on windows perl eats tons of memory. Each
> scintilla instance does so also. Multiple Document handling via STC uses
> less memory.
>
> Anyway, maybe the following (last idea) can solve your problem written at
> the wxperl mailing list:
>
> I found out that setting the codepage of an empty scintilla document changes
> nothing, if you've just created a document. So just try the following to
> ensure that the right document is set and then add the codepage. Just after
> "CreateDocument ..." ...
>
> my $doc   = $self->{Editor}->GetDocPointer;
>
> $self->{Editor}->AddRefDocument($doc);
> $self->{Editor}->SetDocPointer($doc);
> $self->{Editor}->ReleaseDocument($doc);
>
> $self->{Editor}->SetCodePage(65001);
>
> But, this solution only makes sense if you have a STC instance for each
> file. With multiple document views the above will not work ...

Though most of the unicode works (except a few special cases as mentioned
in the report) but I'll try your suggestion.


>
> 2. To reduce crashes ... Ignore SIG{ALARM} which sometimes is send by STC on
> windows with no reason and crashes the whole program ...
>
> $SIG{ALRM}='IGNORE';   # Ignore SIGALRM(14) send by STC

Interesting.


>
> 3. Encode::Byte. I read a conversation between you and another one on IRC in
> the web a few days ago about not recognizing codepage "cp1251". Don't know
> if this is fixed yet at Padre. But I had the same problem and solved it with
> the import of Encode::Byte which holds all codepage constants ...

That's interesting too.


>
> Hope some tips are helpful,
> Greetz Carsten ...


it look so, thanks
  Gabor


>
>
> Gabor Szabo schrieb:
>>
>> Hi Carsten,
>>
>>
>> On Sun, Jul 5, 2009 at 4:25 PM, Carsten Zandecki<service at mediaminds.de>
>> wrote:
>>
>>>
>>> Hi,
>>> i looked a while at your source code of Padre 0.27 (latest version?)...
>>> but
>>> i can't even find the place where you do multiple document handling via
>>> STC.
>>> Don't you change document pointers after switching files over notebook
>>> tabs?
>>>
>>
>> The latest version is 0.38 but soon we will release 0.39
>> For every tab we have a separate STC instance so we don't need to swap
>> them.
>>
>>
>>
>>>
>>> Then I tested Padre 0.27 and everything works fine for me (german umlauts
>>> etc. where displayed correctly also after hitting backspace or del) ...
>>>
>>
>> I checked it again on Ubuntu and 0.27 has the same issues with the
>> Hungarian
>> vowels. What operating system do you run ?
>>
>>
>>
>>>
>>> Btw... there is bug in opening file names with german umlauts (ascii
>>> characters > 127) which could be solved by reading the current system
>>> codepage and decoding it for perls internals.
>>>
>>
>> I'll check this.
>>
>>
>>
>>>
>>> If you don't STC's multiple document handling (do you use threads?)  i
>>> probably can't help and my solution doesn't work.
>>>
>>
>> We also use threads but not for the document handling.
>>
>>
>>
>>>
>>> Also you have to take care about encoding generally. Input/Output ... For
>>> that i convert all data to current systems codepage which works fine (in
>>> my
>>> personal editor), which might be a solution to your problem, too ...
>>>
>>
>> We do that in Padre::Document load_file and save_file
>>
>> Gabor
>>
>>
>>>
>>> Greetz,
>>> Carsten Zandecki ...
>>>
>>> --
>>>
>>>
>>>
>>>
>>>>>
>>>>> Hi there,
>>>>> i remember this behaviour and use the following solution.
>>>>>
>>>>> Herbert Breuning told me to change the codepage to UTF-8, which first
>>>>> doesn't worked for me.
>>>>> But in the meantime I found out that i had to set the codepage for each
>>>>> scintilla document. It is not enough to set codepage only for the
>>>>> scintilla
>>>>> control - i think.
>>>>>
>>>>> So i set the codepage directly after changing the scintilla document
>>>>> pointer
>>>>> (or maybe, it is enough just to set the codepage after creating a
>>>>> document -
>>>>> which i haven't tested yet ... ).
>>>>>
>>>>> This works for me:
>>>>>
>>>>> $self->{Editor}->AddRefDocument($doc);
>>>>> $self->{Editor}->SetDocPointer($self->{Buffer}[$page][0]);
>>>>> $self->{Editor}->ReleaseDocument($self->{Buffer}[$page][0]);
>>>>>
>>>>> $self->{Editor}->SetCodePage(65001);
>>>>>
>>>>> After that you just have to ensure that your file data is UTF-8.
>>>>>
>>>>> Hope it helps ...
>>>>>
>>>>>
>>>>
>>>> We set the code page at the time we create the editor object:
>>>>
>>>>
>>>> http://padre.perlide.org/trac/browser/trunk/Padre/lib/Padre/Wx/Editor.pm#L116
>>>>
>>>> Still after pressing Del I cannot type ő and ű
>>>> other Hungarian vowels can be typed and if I press any key (including
>>>> the shift or Ctrl)
>>>> after that the above two characters can be aslo typed.
>>>>
>>>>
>>>> if you have time, I'd really appreciate if you took a look at Padre and
>>>> how to
>>>> fix this issue there ?
>>>>
>>>> Gabor