Skip to main content
Welcome guest. | Register | Login | Post

Enabling easier translation - changing the structure of GGL

20 replies [Last post]
ariadacapo's picture
Offline
Joined: 2006-07-13

It's time we really change the way the content of GetGNULinux.org is managed.
We have discussed this in the past but in the end I have taken no action. I am now determined to improve this significantly.

I was unsure as to what we needed/wanted, but today I have a much clearer vision. I am simply looking at the case of the Galician translation. The whole GGL content has been translated into Galician on the translation wiki, and has been ready for several weeks now. However, the text has to be laboriously transferred to the XHTML code, while also using our SVN repository. The translator does not have the knowledge to do this, I do not have the time to do it (several days of full-time work, lots of site-specific things to know). This frustrates me a lot, because so much of the useful work has been completed and is just in standstill.
So I think the requirements for a new system are:

  • Easy editing of the content of the pages.
    A user should be able to edit the text of a page (content, not header/menus etc.) with a clear interface without code. The interface must respect any existing tags (XHTML classes, ids...) while allowing editing of the text just like a blog interface.
  • Immediate preview of result.
    There should be no two-step editing (currently wiki then html), one should view what he/she translates in the final form immediately.
  • Headers, footers, CSS, can all be edited manually with code / SVN as they are today. These are rarely changed anyway.
  • Current website appearance and urls entirely unchanged
  • Vadid XHTML code

While working on the planet site I noticed that Wordpress is highly flexible (with the use of php for headers/footers) so I wondered if it could be a solution. I am not really sure about the other possibilities we discussed - the key here is that editing the site should be *easy*. I cannot conceive that one should know and work so much (build code, work with SVN...) just to change a page on our website, while blogs are so easy to build these days.
I look forward to your ideas once again, and this time with a more precise requirements list we should find a solution.
thanks a lot
Olivier.

tbuitenh's picture
Offline
Joined: 2005-12-21
I'm working on a program

I'm working on a program exactly for this kind of task. It will especially be totally awesome for translations!

HTML4 and LaTeX document generation are planned features, XHTML isn't planned but should be trivial.

Unfortunately, I think it won't be finished for the next few months.

Gustavo's picture
Offline
Joined: 2006-09-11
Many of you already know

Many of you already know this Eye ... I suggest DocBook.

I know it's not very easy to setup such a project using DocBook, unless you are used to it. However, once it's up, it will be the easiest, fastest and most powerful way to handle the websites.

Translators would only have to concentrate on the contents and running an app to transform this code into XHTML, PDF, etc. Also, when it comes to doing bulk modifications, this is the best technology. They wouldn't deal with code of a programming language (which I consider risky, btw).

Cheers!

ariadacapo's picture
Offline
Joined: 2006-07-13
How would things happen, if

How would things happen, if we had entirely switched to DocBook Gustavo?

A. How would the header/footer be handled with DocBook? Can it handle a dynamic header?

B. Once set-up, How would the editing of the website happen?

Right now to change a couple of pages, the user has to:
1. update SVN repository
2. find the HTML pages on local computer, edit the HTML code
3. commit changes to SVN
4. upload HTML files to server via FTP, in the right directory
(or alternatively, create a clean copy of the website SVN folder without '.svn' folders, and update the whole website via ftp).

Would it be better with DocBook? Could a user with very little website-related knowledge (ex typical blog user) edit a page easily?
I'm very willing to adopt DocBook but I have no clue how it happens and if it makes things easier (as opposed to more efficient, which I definitely believe it does).

Olivier.

Gustavo's picture
Offline
Joined: 2006-09-11
Hi! A.- The header/footer

Hi!

A.- The header/footer are defined in an external file, content-independent. The contents and the style-related files won't be mixed up. You can apply global changes in one go, at any time.

B- The way to edit files won't change that much, but we'd count with these advantages:

  • Translators won't be encumbered by style-related code. Both translated and original files will be very simpler.
  • We could define the style of all of our websites in an small set of files. If we need to make a bulk modification on all of our websites, we won't have to modify file per file. It's more powerful than CSS/HTML alone.
  • Our URLs won't be broken if we use .htaccess files (as we already do).
  • Starting a translation from scratch and generating its XHTML would be a piece of cake.

If we counted with someone proficient in DocBook, s/he might create the core files only and explain to us how to transfer the contents afterwards.

Cheers!

ariadacapo's picture
Offline
Joined: 2006-07-13
re: docbook

(sorry for delay...)

I am willing to change to DocBook, but is it really going to be worth it?
The main problem right now is not really about editing the site per se - I'm pretty happy with the way it works, I know every single file by heart and can play around easily. SVN is a highly useful system.
What I'm worried about is the fact no-one can participate if they don't learn a lot. The Galician translation is a direct victim of this - and we had quite some trouble with the Catalan translation code too - just because a WYSIWYG application was used. I deeply believe that being able to edit almost directly, with the end-result (finished webpage) displaying instantly, is a crucial feature.
For instance, most of the translations are stalled not because they're hard to edit (MediaWiki makes it really easy) but because what you see on the wiki is very far from the end-result. It's not always motivating for the contributors to edit a wiki rather than a real-looking website.
So DocBook will make the process more efficient, but it might simply turn people off to see they must learn DocBook / compile DocBook to XHTML / use FTP just to see a web page.

In this respect I am most impressed by Wordpress (which I learned to use on the planet). I'm not entirely determined to switch GGL to Wordpress though (I'd lack the control we have over the site today I think).

This is all slightly confused, but I am really afraid we spend a huge amount of time changing to a new high-performance system... only to not actually work better or encourage translations. What do you think?

Gustavo's picture
Offline
Joined: 2006-09-11
Hi! ariadacapo

Hi!

ariadacapo wrote:

(sorry for delay...)

I am willing to change to DocBook, but is it really going to be worth it?
The main problem right now is not really about editing the site per se - I'm pretty happy with the way it works, I know every single file by heart and can play around easily. SVN is a highly useful system.
What I'm worried about is the fact no-one can participate if they don't learn a lot. The Galician translation is a direct victim of this - and we had quite some trouble with the Catalan translation code too - just because a WYSIWYG application was used. I deeply believe that being able to edit almost directly, with the end-result (finished webpage) displaying instantly, is a crucial feature.
For instance, most of the translations are stalled not because they're hard to edit (MediaWiki makes it really easy) but because what you see on the wiki is very far from the end-result. It's not always motivating for the contributors to edit a wiki rather than a real-looking website.
So DocBook will make the process more efficient, but it might simply turn people off to see they must learn DocBook / compile DocBook to XHTML / use FTP just to see a web page.

No, if we use docbook translators won't have to learn anything about docbook nor HTML. They just have to open their favorite text editor, translate the contents previously indicated, save the document and commit it by using svn. At the most, they'll have to run a command to generate the HTML, PDF... Did you remember the shell script I did for us to run it before uploading the websites? Translators would have to run a command like the one we used et voilà, the end work would be in their computers (without uploading it to any server to check for their translations).

ariadacapo wrote:

In this respect I am most impressed by Wordpress (which I learned to use on the planet). I'm not entirely determined to switch GGL to Wordpress though (I'd lack the control we have over the site today I think).

This is all slightly confused, but I am really afraid we spend a huge amount of time changing to a new high-performance system... only to not actually work better or encourage translations. What do you think?

I think we should invest on a better documentation system. We had lots of last-time troubles before releasing FireHippopotamus, which could be avoided or detected earlier if we used a docbook-like infrastructure.

Look at this:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>get GNU/Linux!</title>
<meta name="description" content="Get GNU/Linux! A simple, clear website about Linux.   | What Linux is | Why not use Windows | Tips to make the switch"/>
<meta name="keywords" content="free software, freedom in computing, windows alternative, linux, gnu/linux, get linux, switch to linux"/>
<link rel="shortcut icon" type="image/x-icon" href="files/favicon.ico" />
<link rel="stylesheet" type="text/css" href="files/maincss.css" />
<link rel="stylesheet" type="text/css" href="files/home.css" />

<meta name="generator" content="Bluefish 1.0.6"/>

</head>

<body>
<div class="accessibility"><a href="#maincontent">skip to content</a></div>

<div id="translations" >
<ul>

<li class="other"><a href="http://www.gnulinux.cat/" title="Linux: Una alternativa a Windows, lliure i gratuita" hreflang="ca"> <img src="./files/catalogna30.png" alt="CA" /> </a></li>

<li class="here"><img src="./files/english30.png" alt="EN" /></li>

<li class="other"><a href="http://www.obtengalinux.org/" title="¡Obtenga Linux! Una alternativa a Windows que le da más libertad a sus usuarios." hreflang="es"> <img src="./files/spain30.png" alt="ES" /> </a></li>

<li class="other"><a href="http://www.passeralinux.fr/" title="Linux: Une alternative à Windows, libre et gratuite" hreflang="fr"> <img src="./files/france30.png" alt="FR" /> </a></li>

</ul>
</div><!-- end translations -->

<div id="mainbody">

<div id="header">
<div id="header_picture"><img src="files/getgnulinux.png" width="400" height="130" alt="Get GNU/Linux"/></div>

<ul>
<li><a href="about/">About</a></li>
<li><a href="switch_to_linux/">Making the switch</a></li>
<li><a href="windows/">Why not Windows?</a></li>
<li><a href="linux/">What is Linux?</a></li>
</ul>
</div><!-- end header -->

<div id="maincontent">

<h1>Switch to Linux!</h1>
<a href="linux/" title="What is Linux?"><img width="200" height="226" src="applications.jpg" alt="GNU/Linux applications"/></a>
<p>GNU/Linux, or simply <em>Linux</em>, is an alternative to Microsoft Windows&reg;. It is easy to use and gives more freedom to users. Anyone can install it: Linux is free as in freedom, and often available free of charge.</p>

<div id="content_list">
<ul>

<li><a href="linux/" title="What is Linux exactly?"><strong>What is Linux?</strong><br/>

Learn more about the free operating system</a></li>
<li><a href="windows/" title="What is wrong with Windows?"><strong>Why not Windows?</strong><br/>
Why we should avoid using Microsoft Windows&reg;</a></li>
<li><a href="switch_to_linux/" title="Useful information for switching to linux"><strong>Making the switch</strong><br/>
Where to download and how to step into Linux</a></li>
</ul>
</div><!-- end content_list -->

</div><!-- end maincontent -->
</div><!-- end mainbody -->

<div id="footer">
<span class="link_button"><a href="http://www.getfirefox.com/" title="Get Firefox - a free web browser that changes the web"><img src="http://www.getgnulinux.org/link/firefox_button.png" width="80" height="15" alt="| get Firefox |"/></a></span>
<span class="link_button"><a href="http://www.getgnulinux.org/" title="Get Linux - an alternative to Windows; free as in beer and speech"><img src="http://www.getgnulinux.org/link/getgnulinux_80x15_1.png" width="80" height="15" alt="| get Linux |"/></a></span>

<span id="about"><a href="http://www.getgnulinux.org/about/" title="Learn about GetGNULinux.org">About us</a></span>
<span id="legal"><a href="http://www.getgnulinux.org/about/legal/" title="Legal information about the contents of this website">Legal Info</a></span>
</div><!-- end footer -->

</body>
</html>

... and compare it to this:

<title>get GNU/Linux !</title>

<h1>Switch to Linux !</h1>
<a href="linux/" title="What is Linux?"><img width="200" height="226" src="applications.jpg" alt="GNU/Linux applications"/></a>
<p>GNU/Linux, or simply <em>Linux</em>, is an alternative to Microsoft Windows&reg;. It is easy to use and gives more freedom to users. Anyone can install it: Linux is free as in freedom, and often available free of charge.</p>

<ul>
<li><a href="linux/" title="What is Linux exactly?"><strong>What is Linux?</strong>
Learn more about the free operating system</a></li>
<li><a href="windows/" title="What is wrong with Windows?"><strong>Why not Windows?</strong>
Why we should avoid using Microsoft Windows&reg;</a></li>
<li><a href="switch_to_linux/" title="Useful information for switching to linux"><strong>Making the switch</strong>
Where to download and how to step into Linux</a></li>
</ul>

The former is our current main page's code, which could be generated from the second code. If we used a docbook-like infrastructure, the text we edited would look like the second code, which would be quite easy to edit and avoid mistakes as we won't see so much style-related code.

Where is the rest of the code? How is the computer going to know the code it has to add? The rest of the code would be declared once and in a single file, which translators won't have to modify.

While translators would have to use a text editor, the command line (to run a single command), a web browser and a svn client, it's easier than dealing with wiki code or HTML. Also, if they make a mistake, they'd detect it quick and easily... And we'll always have valid XHTML.

I think this approach is far easier for translators, but harder for us to set it up. As I already said, this would be a one-time effort for a long-term benefit.

Cheers.

tbuitenh's picture
Offline
Joined: 2005-12-21
Since I'm writing a

Since I'm writing a document translation web app, I hope my words contain some wisdom...

I don't think the editor should be WYSIWYG (which would cause all kinds of trouble for people using non-standard browsers), but the following things are important:

- if you click "save", you will be shown something that looks more or less like the final webpage, so you can immediately see your mistakes

- you can do a paragraph-by-paragraph comparison of the translation and the original, or two translations

- while comparing, you can also edit both original and translation (if you have proper permissions)

- changes to the original (or other translation) are easy to find

Will the above be true for my app? Yes sir!
Does it already do any of the above? No, I'm currently busy writing a friendly yet cracker-proof login and user administration system, the interesting bits can only be built later.

Does docbook have any of the above features? Nope. So I don't really see the point of using it. It will be useful when you also export to other formats than just xhtml, though.

ariadacapo's picture
Offline
Joined: 2006-07-13
WP vs DocBook

Hmmm... it's really difficult to decide here. I can start to see the main advantage of DocBook is that it can be edited in the translator's offline application.
But as tbuitenh pointed out, it does not allow immediate preview of the result. Too bad your system is not up and running, tbuitenh! ;-)

In any case, we will use a system where only the page content must be edited, as you show Gustavo. The header/footer/etc must be generated automatically. This is a feature supported by Wordpress, DocBook or most other systems... we might have some trouble with the translation tabs (they are unique to each page) but this is a side issue.

I have taken a printscreen of how it would work with Wordpress, using the current install on the planet:
http://test.getgnulinux.org/temp/wppage.png (~250K)
Note that the user only has to use the page content, edit the text visually (or switch to code view, if the page is more complex, ex Try and Install page). He/she then has a preview of the end-result, including header+footer.

I'm totally undecided and hesitating about the problem.

I'm certain DocBook is more efficient, however if I understand correctly, a typical DocBook editing process looks like this:
+ update SVN to get DocBook files
+ edit DocBook file (easy, can be done in editor without handling code)
+ run script to generate HTML
+ check HTML in browser
+ correct DocBook file if needed
+ commit files to SVN, both the HTML and DocBook
+ clear up HTML files from SVN information with simple script
+ upload HTML files to FTP

A typical WordPress editing process looks like this:
+ login to WP (online)
+ edit HTML (some pages will be easy -only WYSIWYG, like "What is Linux", some much harder, have to handle HTML code, like "Try and Install")
+ check preview right below edit area
+ correct code immediately
+ save. Pages are instantly changed online.

So from my point of view the balance is
DocBook

pros
*Very good for long-term. We will have a very clear separation of content and style. We can change the site system anytime.
*Lots of control, over files, users, etc; SVN enables easy verification by anyone.
*Users edit content with very little code

cons
*not much simpler than actual situation really. Huge amount of learning required for non-experienced user.
*process for just correcting a typo is pretty long!

Wordpress

pros
*Right damn easy. What could be simpler?

cons
*Risk that pages with lots of tags (ex Try and Install page) are not handled properly if user attempts to use only WYSIWYG
*No real control over who does what (as SVN could permit). User can edit a file with code or content error immediately
*Site might be much slower (dynamic pages)
*Probably hard to export our content if we want to switch to something else later.

Am I getting this right?

Gustavo's picture
Offline
Joined: 2006-09-11
I answered you ~3days ago

Yes, that's right!

I answered you ~3days ago and Drupal told me that my post seemed to be spam and that it should be accepted by the admin before it be published.

I don't why it's not published yet (it was a long post, btw)... I'll contact libervisco.

Cheers!

EDIT: Actually, I answered your previous post.

libervisco's picture
Offline
Joined: 2006-05-04
My apologies. It's

My apologies. It's published now.

There was lots of spam being detected on both sites so this was lost on me as a false positive.

Thanks

tbuitenh's picture
Offline
Joined: 2005-12-21
You could make svn+docbook

You could make svn+docbook usage easier by writing a few scripts. I would recommend a makefile with targets update, edit, test and commit, where edit depends on update (and launches either $EDITOR or $VISUAL), test runs docbook and launches $BROWSER, and commit depends on test (and asks if the test was ok). So a typical session would look like this:

export TOEDIT=introduction
make edit (launches favorite editor)
make commit (launches browser)
-- OK to commit?
n
make edit
make commit
-- OK to commit?
y
export TOEDIT=whynotwindows
make edit
...

I could easily add docbook export to my software, but docbook import is far from trivial because docbook has too many features, yet, as far as I know, doesn't have support for my inventions (strange, isn't that? Eye ) which my software depends on to prevent conflicts.

If you make sure translations keep the same order of paragraphs as the original, and are complete, we could do a semi-automated one time import when my software is finished and tested.

Gustavo's picture
Offline
Joined: 2006-09-11
Don't worry! Thanks!

Don't worry! Thanks!

Gustavo's picture
Offline
Joined: 2006-09-11
I like it!But, don't you

I like it!

But, don't you think we are better off transferring the contents by hand? I mean, right now we have 5 websites and I don't know if creating such a tool to automate the transference might take more time than doing this by hand. If I'm mistaken, then this is not a problem.

Let's see what others have to say.

Cheers!

tbuitenh's picture
Offline
Joined: 2005-12-21
With semi-automated I mean

I'm not really sure which part of my post you are referring to.

The makefile would be just a few lines.

With semi-automated import I mean using just a small python script. It would be less (or at least less boring) work than going through some web forms a few times for each page of each translation. Anyway, what is the easiest solution really depends on how simple your data structures are. We'll see.

Gustavo's picture
Offline
Joined: 2006-09-11
Yes, the semi-automated

Yes, the semi-automated import. Now I get it, so I think it's okay.

Cheers!

ariadacapo's picture
Offline
Joined: 2006-07-13
Miguel's comment

Just wish to quote kuan aka Miguel from the mailing-list on the topic:

kuan wrote:

I have been following the discussion about GGL structure.
I had some spare time so I knocked up a proof of concept for how this could work.

To sum this up, problems and advantages of:

[Static]
+ Easily customizable
+ Forget about spam
+ Fast and safe
- Hard to maintain
- It does not allow user interaction
[Dynamic]
+ Documents can be edited online
+ It can be updated more regularly
+ More attractive interface
- Spammable
- Slower than html pages

In any case, short urls is a must have.

It is a good idea to separate content and presentation with XML.
The workflow would be like this:
Edit xml -> upload files to svn -> verify them -> publish them.
If we change of mind about how to manage the content, we would be able to export it.

We can use a xml-based cms to make this faster.
Bitflux (http://www.flux-cms.org/) has everything that Gustavo talked about, including i18n support.

Another possibility would be the wiki way.
DokuWiki(http://wiki.splitbrain.org/wiki:dokuwiki) is a pretty complete option. It does not need a database, and many features can be added through plugins.
Controlling users can be done with access control lists. Correct me if I am wrong, I think the Xfce.org site uses it.

I am a cli fan, so I like the method that tbuitenh suggested. However, the easiest solution depends on how the different domains will be managed.

ariadacapo's picture
Offline
Joined: 2006-07-13
The key question

The key question we need to answer, I think, is whether we want to use SVN. All the rest depends on this.

Is the functionality of SVN (see who changed what) important enough for us?
If yes then we should probably go for the solution tbuitenh proposed
If no then we can pick a CMS / blog server-side software with good user-control.

I believe the ease of translation takes precedence - I would trade off advanced control, good separation of content/style (which we get with any SVN-based solution, including docbook/XML), for simplicity (which we get with online-editing). There are many, many languages out there and each new translation widely increases the audience we can reach. Keeping the translation process within reach of an average ubuntu user, to me, is a major advantage: many translations have been started but few have substantial work completed, because I failed to provide an easy way to do things.
What do you think? Is this analysis wrong?

Gustavo's picture
Offline
Joined: 2006-09-11
I think svn is absolutely

I think svn is absolutely necessary. It's the best way to keep track on what's exactly going on.

By the way, I'm afraid there exists xslt-capable browsers; this would mean that we can modify the files and see the result instantaneously, with no previous transformation done by hand. If I'm right, I think DocBook would be the way to go. [1] [2].

OTH, I think you're right about how difficult is to translate ggl into other languages at this moment.

Cheers!

ariadacapo's picture
Offline
Joined: 2006-07-13
Let's try it out then! I

Let's try it out then! I suggest we build a test page with Docbook/SVN .

I have made a test page at http://test.getgnulinux.org/test/ that represents one of the most complicated pages of GGL. For now the page is simply made by adding simple php elements (without any programming, just the include function).

We have 3 things to do for the system to work:

1. Construct a Docbook file that can export the content of the page (content.php).

2. Construct the header dynamically with PHP (for example the current section can be highlighted dynamically) so that the same header.php file can be used for the whole website

3. Make it easy to combine Docbook, constructed HTML and SVN, as perhaps described by tbuitenh.

The files are available on our "useful" repository: http://svn.getgnulinux.org/useful You can login there with test/test or (much better) email me for a proper login.
Ask me per email for the password of the FTP for uploading to http://test.getgnulinux.org/test/

This is far beyond my knowledge (I handle XHTML&CSS allright, but know no PHP or Docbook) so I'll need lots of help on this.
Let's try this out. If we can handle this single page, then switching the whole website will be rather easy.
Thanks
Olivier.

Gustavo's picture
Offline
Joined: 2006-09-11
Hi, Olivier.PHP is not

Hi, Olivier.

PHP is not required; the rest is alright.

I've been (slowly) setting up the first steps website using DocBook *, so first we might use it as our guinea pig to find out whether it works for our projects, as well as in order to learn about it.

Olivier, you might want to read the introduction to DocBook; I did it and it's a good source to get us started with this technology, imo.

* I'm going to create an svn repo to host what I've done so far; more on this soon.

What do you think about this?

Cheers!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.