abiword Related Pages

Application Framework

The 'af' directory contains all source code for the cross-platform application framework. This directory contains the following subdirectories:

Directory ev

Source code for the event mechanism for the cross-platform application framework. This code contains the machinery to do key bindings, mouse bindings, menu bars, and tool bars. This code is currently used by the word processor, but we expect to also use it for the spreadsheet application.

Directory gr

Source code for graphics (drawing code, font code, etc.)

Directory util

Source code for general-purpose utility functions.

Directory xap

Source code for application-neutral portion of the cross-platform framework defined in ev.

Squiggles

Squiggles are used to underline miss-spelled words. Instead of simply erasing all squiggles and rechecking all words in a block when the block is changed, the squiggles are handled much like the words they underline.

The word currently being edited is the pending word. When the cursor does not touch the pending word anymore (due to being moved away or the user typing a word separator), the word is spell-checked. If it is miss-spelled, it will be squiggled.

When text is added to the block, fl_Squiggles::textInserted is called with information of where in the block the text was added, and how much. It will then remove any squiggle located at that offset, move all following squiggles (so they end up aligned with the words they should underline) and spell-checks words in the added text (via fl_BlockLayout::_recalcPendingWord).

When text is deleted from the block, fl_Squiggles::textDeleted is called with information of where in the block text was deleted, and how much. It removes squiggles intersecting with that area, moves all following squiggles and makes pending the word at the deletion point since two words may have been joined, or a word lost part of its letters.

When a block is split in two, fl_Squiggles::split is called with information of where the block was split, and a pointer to the new block. The squiggles from the old block are split between it and the new block. The word at the end of the old block (which may have been broken), is spell-checked, and the first word of the new block is made the pending word.

When two blocks are merged into one, fl_Squiggles::join is called with information of the offset where squiggles from the second block should be joined onto the first block. The word at the merge point is made the pending word.

There's one known buglet: typing "gref' " correctly squiggles the word when typing ' since it's a word separator. However, deleting the s in "gref's" leaves gref unsquiggled because the word gref was not pending when the ' changed from a word character to a word delimiter. (hard to explain - just try it)

Formatter

FL_DocLayout is a formatted representation of a specific PD_Document, formatted for a specific GR_Graphics context.

A FL_DocLayout encapsulates two related hierarchies of objects.

The logical (or content) hierarchy corresponds to the logical structure of the document.

• each FL_DocLayout contains a list of fl_SectionLayout objects
• each fl_SectionLayout are composed of fl_BlockLayout objects

Where each fl_BlockLayout corresponds to a logical element in the PD_Document (i.e., usually a paragraph of text).

The physical (or layout) hierarchy, by contrast, encapsulates the subdivision of physical space into objects of successively finer granularity.

• each FL_DocLayout contains a list of fp_Page objects, each of which was constructed during the process of formatting the document
• each fp_Page is a list of fp_Columns (fp_Column)
• each fp_Column is a list of fp_Lines (fp_Line)
• each fp_Line is a list of fp_Runs (fp_Run)

Where each fp_Run contains some fragment of content from the original document, usually text.

Other subjects

• Squiggles used by the spell checker.

PieceTable

Todo:

Add more class names / links to sources.

1. PieceTable

1.1. Introduction

A pt_PieceTable is the data structure used to represent the document. It presents an interface to access the document content as a sequence of (Unicode) characters. It includes an interface to access document structure and formatting information. It provides efficient editing operations, complete undo, and crash recovery.

1.2. Class Overview

The PieceTable consists of the following classs:

1. InitialBuffer -- This is a read-only character array consisting of the entire character content of the document and initially read from the disk. (All XML tags and other non-content items are omitted from this buffer.)

2. ChangeBuffer -- This is an append-only character array consisting of all character content inserted into the document during the editing session.

3. InitialAttrPropTable -- This is a read-only table of Attribute/Property structures extracted from the original document.

4. ChangeAttrPropTable -- This is an append-only table of Attribute/Property structures that are created during the editing session.

5. Piece -- This class represents a piece of the sequence of the document; that is, a contiguous sub-sequence having the same properties. Such as a span of text or an object (such as an in-line image). It contains a links to the previous and next Pieces in the document. Pieces are created in response to editing and formatting commands.

   1. TextPiece -- This subclass represents a span of contiguous text in one of the buffers. All text within the span has the same (CSS) properties. A TextPiece is not necessarily the longest contiguous span; it is possible to have adjacent (both in order and in buffer position) TextPieces with the same properties. A TextPiece contains a buffer offset and length for the location an size of the text and a flag to indicate which buffer. A TextPiece contains (or contains a link to) the text formatting information. Note that the buffer offset only gives the location of the content of the span in one of the buffers, it does not specify the absolute position of the span in the document.

   2. ObjectPiece -- This subclass represents an in-line object or image. It has no references to the buffers, but does provide a place-holder in the sequence.

   3. StructurePiece -- This subclass represents a section or paragraph. It has no references to the buffers, but does provide (CSS) style information and a place-holder in the sequence. There are no links between StructurePieces or between other Pieces and their (containing) StructurePieces.

6. PieceList -- This is doubly-linked list of Pieces. The are linked in document order. A forward traversal of this list will reveal the entire content of the document; in doing so, it may wildly jump around both of the buffers, but that is not an issue.

7. PX_ChangeRecord -- Each editing and formatting change is represented as a ChangeRecord. A ChangeRecord represents an atomic change that was made to one or more pieces. This includes offset/length changes to a TextPiece and changes to the PieceList.

8. ChangeVector -- This is a vector of ChangeRecords. This is used like a stack. ChangeRecords are appended to the vector (pushed onto the stack) as they are created in response to editing and formatting commands. The undo operation takes the last ChangeRecord in the vector and un-does its effect. A redo operation re-applies the ChangeRecord. The ChangeVector holds the complete information to undo all editing back to the initial document. The index of the current position in the ChangeVector is maintained. ChangeRecords are not removed from the vector until the redo is invalidated. When a ChangeRecord is removed from the vector, it is deleted.

1.3. Operations

1. Insert(position,bAfter,c) -- To insert one or more characters c into the document (either before or after) the absolute document position position, we do the following:

   1. Append the character(s) to the ChangeBuffer.
   2. Find the TextPiece that spans the document position.
      • If the document position is in the middle of a TextPiece (p1), we split it into two TextPieces (p1a, p1c) and create a third TextPiece (p1b). p1a and p1c contain the left and right portions referenced in p1. p1b spans the newly-inserted character(s). The PieceList is updated so that the sequence p1a,p1b,p1c replace p1 in the list.
      • If the document position is at the end of a TextPiece and the buffer position in either buffer is contiguous with the buffer and position referenced in the TextPiece and the formatting is the same, we may avoid the three part split and simply update the offset/length in the TextPiece. This case is very likely when the user is composing text or is undoing a delete.
      • If the document position is between Pieces, a new TextPiece is created and inserted into the PieceList.
   3. Create a ChangeRecord and append it to the ChangeVector. For an insert, we construct a ChangeRecord of type InsertSpan.
      • cr.span.m_documentOffset contains the document position of the insertion.
      • cr.span.m_span marks the buffer position of the text that was inserted.
      • cr.span.m_bAfter remembers whether the insertion was before or after the document position.

2. Delete(position,bAfter,length) -- To delete one or more characters from the document (either before or after) the absolute document position position, we do the following:

   1. Find the TextPiece that spans the document position.
      • If the length of characters is contained within the TextPiece (p1), we split it into two TextPieces (p1a and pl1b). The offsets and lengths are set in the new TextPieces such that the deleted sequence is not in either piece. (The deleted text is not actually deleted from the buffer; there are just no references to it from the PieceList.)
      • If the document position is at the beginning or end of a TextPiece, we can just adjust the offset/length, rather than doing the split.
      • If the deletion extends over multiple Pieces, we iterate over each piece in the range and perform a delete on the sub-sequence. This will result in a multi-step ChangeRecord.
      • TODO what about non-TextPieces??
   2. Create a ChangeRecord and append it to the ChangeVector. For a delete, we construct a ChangeRecord of type DeleteSpan.
      • cr.span.m_documentOffset contains the document position of the deletion.
      • cr.span.m_span marks the buffer position of the text that was deleted.
      • cr.span.m_bAfter remembers whether the insertion was before or after the document position.
3. InsertFormatting()

4. ChangeFormatting()

5. Undo -- This can be implemented using the information in the ChangeVector. If the CurrentPosition in the ChangeVector is greater than zero, we have undo information. The information in the ChangeRecord prior to the CurrentPosition is used to undo the editing operation. After an undo the CurrentPosition is decremented.

   • If the ChangeRecord is of type InsertSpan: we perform a delete operation using cr.span.m_documentOffset, cr.span.m_span.m_length and cr.span.m_bAfter.

   • If the ChangeRecord is of type DeleteSpan: we perform an insert operation using cr.span.m_documentOffset, cr.span.m_span, and cr.span.m_bAfter.

   • If the ChangeRecord is of type ChangeFormatting:
   • If the ChangeRecord is of type InsertFormatting:

6. Redo -- This can be implemented using the information in the ChangeVector. If the CurrentPosition in the ChangeVector is less than the length of the ChangeVector, the redo has not been invalidated and may be applied. The information in the ChangeRecord at the CurrentPosition provides complete information to describe the editing operation to be redone. After a redo the CurrentPosition is advanced.

7. Autosave -- This can be implemented by periodically writing the ChangeBuffer, ChangeVector, and the ChangeAttrPropTable to temporary files. After a crash, the original document and the temporary files could be used to replay the editing operations and reconstruct the modified document.

1.4. Observations

1. The content of the original file are never modified. Pieces in the PieceList describe the current document; the original content is referenced in a random access fashion. For systems with small memory or for very large documents, it may be worth demand loading blocks of the original content rather than loading it completly into the InitialBuffer.

2. Document content data (in the two buffers) are never moved once written. insert and delete operations change the Pieces in the PieceList, but do not move or change the contents of the two buffers.

3. The result of an undo operation must produce the identical document structure and content. Since consecutive Pieces in the PieceList may have the same formatting properties and may refer to congituous buffer locations (there is no requirement to coalesce them), an undo operation may produce a different PieceList than we originally had prior to doing the operation that was undone.
   • TODO Check this. Whether the PieceList should be identical or equivalent.

1.5. Problems or Issues

1. TextPieces represent spans of text that are convenient for the structure of the document and a result of the sequence of editing operations. They are not optimized for layout or display.

   • We can provide access methods to return a const char * into the buffers along with a length, which the caller could use in text drawing or measuring calls, but not c-style, zero-terminated strings.

2. Mapping an absolute document position to a Piece involves a linear search of the PieceList to compute the absolute document position and find the correct Piece. The number of Pieces in a document is a function of the number of editing operations that have been performed in the session and of the complexity of the structure and formatting of the original document. A linear search might be painfully slow.

   • TODO We have a patch to use an rbtree instead of the doubly-linked list to give us O(log(n)) searching, but the memory consumption is still terrible and there is much improvement to be made. It is recommended that we optimize away the color and better yet, just switch to a fractal prefetching b+-tree. Other ideas include a topologically integrated mesh data structure, but the implementation and wrapping (for sanity's sake) of this would be quite a bit more work than the fpbtree.
   • TODO Consider caching the last few lookup results so that we can avoid doing a search if possible. This should have a high hit-rate when the user is composing text.

3. We provide a complete, but first-order undo with redo. That is, we do not put the undo-operation in the undo (like emacs).

4. TODO The before and after stuff on insert and delete is a bit of a hand-wave.

5. TODO Need to add multi-step-undo so that delete operations which span multiple pieces can be represented operation to the user.

1.6. Code

class PT_PieceTable
{
	const UT_UCSChar * m_InitialBuffer;
	const UT_UCSChar * m_ChangeBuffer;
	pt_PieceList * m_pieceList;
	pt_AttrPropTable m_InitialAttrPropTable;
	pt_AttrPropTable m_ChangeAttrPropTable;
	...
};

class pt_Piece
{
	enum PieceType	{ TextPiece,
			  ObjectPiece,
			  StructurePiece };
	PieceType	m_pieceType;
	<linked-list or tree pointers>
	...
};

class pt_Span
{
	UT_Bool		m_bInInitialBuffer;
	UT_uint32	m_offset;
	UT_uint32	m_length;
};

class pt_TextPiece : public pt_Piece
{
	pt_Span			m_span;
	pt_AttrPropReference	m_apr;
	...
};

class pt_ObjectPiece : public pt_Piece
{
	...
};

class pt_StructurePiece : public pt_Piece
{
	pt_AttrPropReference	m_apr;
	...
};

class pt_PieceList
{
	<container for linked-list or tree structure>
	...
};

class pt_AttrPropReference
{
	UT_Bool		m_bInInitialTable;
	UT_uint32	m_index;
	...
};

class pt_AttrProp
{
	UT_HashTable * m_pAttributes;
	UT_HashTable * m_pProperties;
	...
};

class pt_AttrPropTable
{
	UT_vector<pt_AttrProp *>	m_Table;
	...
};

class pt_ChangeRecord
{
	UT_Bool m_bMultiStepStart;
	UT_Bool m_bMultiStepEnd;

	enum ChangeType	{ InsertSpan,
			  DeleteSpan,
			  ChangeFormatting,
			  InsertFormatting,
			  ...
			};
	struct {
		UT_uint32	m_documentOffset;
		UT_Bool		m_bAfter;
		pt_Span		m_span;
	} span;
	struct {
		UT_uint32	m_documentOffset1;
		UT_uint32	m_documentOffset2;
		pt_AttrPropReference	m_apr;
	} fmt;
	...
};

class pt_ChangeVector
{
	UT_vector	m_vecChangeRecords;
	UT_uint32	m_undoPosition;
	...
};

Text

The 'text' directory contains the text-editing engine used by AbiWord and other AbiSuite apps. There is one subdirectory per module.

Directory fmt/xp (Formatter):

Contains formatting and layout code, including views.

Directory ptbl/xp (PieceTable):

Contains the editable document, implemented using piece tables.

ImpExp

Todo:

Finish it.

1. Generalities

This part contains all the importer and exporter code used by AbiWord. IE_Imp_* classes are the document importers. IE_Exp_* classes are exporters. IE_ImpGraphic_* classes are graphics importers.

Importers and exporters are also used for clipboard operations.

2. Importers

• IE_Imp -- This is the base class for all WP importers.

• IE_Imp_AbiWord_1 -- Imports version 1 (ie current) of AbiWord documents

• IE_Imp_Applix -- This is the importer for Applix Words documents.

• IE_Imp_DocBook -- Importer for DocBook SGML documents.

• IE_Imp_GraphicAsDocument -- Import a graphic as an empty document containing that graphics. Use available IE_ImpGraphic_*

• IE_Imp_GZipAbiWord -- Imports gzip compressed AbiWord documents (.zabw)

• IE_Imp_MsWord_97 -- Imports MS Word 97 documents using libwv.

• IE_Imp_RTF -- This is the RTF importer.

• IE_Imp_Text -- Plain text importer. Also handle non-ASCII text.

• IE_Imp_WordPerfect -- Imports WordPerfect documents.

• IE_Imp_XHTML -- Import valid XHTML documents.

• IE_Imp_XML -- Generic XML importer. Used as a base class for all other XML work.

3. Graphic Importers

• IE_ImpGraphic -- This is the base class for all graphics importers.

• IE_ImpGraphic_JPEG -- This is the JPEG importer using jpeglib. Convert JPEG image to a PNG image.

• IE_ImpGraphic_PNG -- This is the PNG importer. Simply reads the PNG file.

• IE_ImpGraphic_BMP -- This is the BMP importer. Convert a BMP file to PNG.

• IE_ImpGraphic_WMF -- WMF Importer.

• IE_ImpGraphic_SVG -- SVG Importer. Currently worthless.

4. Exporters

• IE_Exp_AbiWord_1 -- Write AbiWord XML files version 1, the native file format as of today.

• IE_Exp_AWT -- Write AbiWord template documents. Most of the functionnality is inherited from IE_Exp_AbiWord_1

• IE_Exp_HTML -- Output HTML 4.0 or XHTML.

• IE_Exp_RTF -- Exports RTF.

• IE_Exp_Text -- Exports plain text.

WP

The 'wp' directory contains all source code specific to AbiWord. There is one subdirectory per module.

Directory ap (AP)

Contains source code for application-specific portion of the cross-platform framework defined in src/af/xap and src/af/ev. This contains application key bindings, mouse bindings, menu layouts, and toolbar layouts. It contains the menu string tables. It contains the table of application functions to which events may be bound. It contains the code to manage the document window (rulers, scroll bars, and the actual document window itself).

Directory impexp (ImpExp)

Contains importers and exporters for various file formats.

Directory main (main)

Contains platform-specific source code for main().

Subdirectories below may have additional hierarchy to further break things down by module. However, eventually, source code should find itself in a directory which indicates the portability of the code within it. For example, cross-platform code should always be placed in a subdirectory called 'xp'. Win32-specific code should be in a subdirectory called 'win'.

Todo List

Member AP_CocoaApp::getPrefsValueDirectory (bool bAppSpecific, const gchar *szKey, const gchar **pszValue) const

support meaningful return values.

Member AP_CocoaApp::getStringSet (void) const

This function should be inilined.

Member AP_CocoaApp::shutdown (void)

The return value should be fixed to check the return values of the functions it calls, and potentially handle errors. At a minimum, it should return false on errors.

Member AP_UnixApp::getPrefsValueDirectory (bool bAppSpecific, const gchar *szKey, const gchar **pszValue) const

support meaningful return values.

Member AP_UnixApp::getStringSet (void) const

This function should be inilined.

Member AP_UnixApp::pasteFromClipboard (PD_DocumentRange *pDocRange, bool bUseClipboard, bool bHonorFormatting=true)

currently i have this set so that a ^v or Menu[Edit/Paste] will use the CLIPBOARD property and a MiddleMouseClick will use the PRIMARY property -- this seems to be the "X11 way" (sigh). consider having a preferences switch to allow ^v and Menu[Edit/Paste] to use the most recent property... this might be a nice way of unifying things -- or it might not -- this is probably an area for investigation or some usability testing.

Member AP_UnixApp::shutdown (void)

The return value should be fixed to check the return values of the functions it calls, and potentially handle errors. At a minimum, it should return false on errors.

Member AP_UnixToolbar_StyleCombo::getPangoAttrs (PD_Style *pStyle, PangoFontDescription *desc)

ROB parse more attributes like font-color, background-color

Member fp_CellContainer::setWidth (UT_sint32 iWidth)

Should force re-line-break operations on all blocks in the container

Member fp_VerticalContainer::insertContainerAfter (fp_Container *pNewContainer, fp_Container *pAfterContainer)

This function has been hacked to handle the case where pAfterContainer is NULL. That case should not happen. Bad callers should be identified and fixed, and this function should be cleaned up.

Member fp_VerticalContainer::setWidth (UT_sint32)

Should force re-line-break operations on all blocks in the container

Member FV_View::getEditableBounds (bool bEnd, PT_DocPosition &docPos, bool bOverride=false) const

speed this up by finding clever way to cache the size of the header/footer region so we can just subtract it off.

Member FV_View::isImageSelected (void) const

eventually make it faster by not fetching the image data ID.

Member GR_CairoGraphics::polygon (UT_RGBColor &c, UT_Point *pts, UT_uint32 nPoints)

Rob find out how to have this function used, and test.

Member IE_Imp_Applix::_applixNewPara (const char *buf, size_t len)

TODO handle the style and paragraph attributes.

Member IE_Imp_Applix::_applixPageBreak (const char *buf, size_t len)

TODO handle even/odd page, currently ignored by Abiword

Member IE_Imp_RTF::LoadPictData (PictFormat format, const char *image_name, struct RTFProps_ImageProps &imgProps, bool isBinary=false, long binaryLen=0)

TODO: We assume the data comes in hex. Check this assumption as we might have to handle binary data as well

Page ImpExp

Finish it.

page Main Page

Add more class names / links to sources.

Page PieceTable

Add more class names / links to sources.

Member RTF_msword97_level::ParseLevelText (const std::string &szLevelText, const std::string &szLevelNumbers, UT_uint32 iLevel)

look up the parent label and be more precise about what is added by this label.

Class RTFHdrFtr

add right and left headers and footer. Not yet supported by AbiWord

Member UT_convert (const char *str, UT_sint32 len, const char *from_codeset, const char *to_codeset, UT_uint32 *bytes_read, UT_uint32 *bytes_written)

Check for out-of-memory allocations etc.

Bug List

Member AP_CocoaApp::initialize (void)

This function is 136 lines - way too long. Needs to be refactored, to use a buzzword.

Member AP_UnixApp::initialize (bool has_display)

This function is 136 lines - way too long. Needs to be refactored, to use a buzzword.

Member fp_Run::_drawTextLine (UT_sint32, UT_sint32, UT_uint32, UT_uint32, UT_UCSChar *)

Currently, this does not detect whether it is on the screen or not, so it redraws way too often.