当前位置：网站首页>PostgreSQL source code (58) tuple splicing heap_ form_ Tuple analysis

PostgreSQL source code (58) tuple splicing heap_ form_ Tuple analysis

2022-07-01 23:28:00 【mingjie73】

edition ：14
relevant ：
《Postgresql Source code （51） Variable length type implementation (valena.c)》
《Postgresql Source code （56） Extensible type analysis ExpandedObject/ExpandedRecord》

2 background

  • PG There are two forms of representation of the metagroup ：expanded Format （ Easy to calculate ） and flatten Format （ Easy to save ）
  • above 《Postgresql Source code （56） Extensible type analysis ExpandedObject/ExpandedRecord》 The extended format of tuples is described in
  • This article introduces the more general of tuples flatten Format HeapTupleData
  • expanded Format and flatten Formats can be converted to each other （flatten_into A function pointer , Reference resources Postgresql Source code （56））
typedef struct HeapTupleData
{
    
	uint32		t_len;			/* length of *t_data */
	ItemPointerData t_self;		/* SelfItemPointer */
	Oid			t_tableOid;		/* table the tuple came from */
	HeapTupleHeader t_data;		/* -> tuple header and data */
} HeapTupleData;

3 HeapTuple Constructor for heap_form_tuple

HeapTuple The structure is heap_form_tuple Splicing in function , The following article focuses on the analysis of this function ：

Inserted here 5 Take column data as an example ： Three fixed lengths 、 Second, it becomes longer 

drop table t21;
create table t21(i1 int, v10 varchar(10), n1 numeric, c2 char(2), t1 text);
insert into t21 values (1, 'mylen=7', 5.5, '22', 'hi12345');

3.1 heap_form_tuple Enter the reference

Constructors heap_form_tuple

HeapTuple
heap_form_tuple(TupleDesc tupleDescriptor, Datum *values, bool *isnull)

Note that the input parameter is a tuple descriptor 、 An array of values 、isnull Array , The value array contains int Value or datum Data pointer 

(gdb) p *tupleDescriptor
$9 = {natts = 5, tdtypeid = 2249, tdtypmod = -1, tdrefcount = -1, constr = 0x0, attrs = 0x199ce90}
(gdb) p values[0]
$11 = 1            ： int Value 
(gdb) p values[1]
$12 = 27157600     ： datum Data pointer 
(gdb) p values[2]
$13 = 27153160     ： datum Data pointer 
(gdb) p values[3]
$14 = 27158432     ： datum Data pointer 
(gdb) p values[4]
$15 = 27154592     ： datum Data pointer 
(gdb) p isnull[0] 
$17 = false
(gdb) p isnull[1]
$18 = false
(gdb) p isnull[2]
$19 = false
(gdb) p isnull[3]
$20 = false
(gdb) p isnull[4]
$21 = false

3.2 heap_form_tuple Execute the process

  • Be careful ：hoff The position is HeapTupleHeaderData In the future, it can be shifted to data
  • Be careful ：tuple->t_data The position is HeapTupleData How much can you offset back to HeapTupleHeaderData The position of the head
  • The memory structure is ：HeapTupleData+HeapTupleHeaderData+ data 
heap_form_tuple
...
    len = offsetof(HeapTupleHeaderData, t_bits)        ：  Calculate the size of the head len = 23,t_bits Is a flexible array pointer 
    hoff = len = MAXALIGN(len);                        ：  alignment hoff = len = 24
    data_len = heap_compute_data_size(...)             ：  See 3.3, common data_len = 30 byte 
    len += data_len;                                   ： len = 24 + 30 = 54

    tuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len)   ：  apply HeapTupleData + HeapTupleHeaderData +  data 30 byte 
    tuple->t_data = td = (HeapTupleHeader) ((char *) tuple + HEAPTUPLESIZE)
                                                       ： t_data Pointing to HeapTupleData after ,HeapTupleHeaderData The position of the head 
    ...
    //  To configure tuple Value 
    ...
    heap_fill_tuple                                    ：  Start adding data according to the data type , see 3.4

3.3 heap_compute_data_size

Calculate the data length heap_compute_data_size, Already below SQL For example 

drop table t21;
create table t21(i1 int, v10 varchar(10), n1 numeric, c2 char(2), t1 text);
insert into t21 values (1, 'mylen=7', 5.5, '22', 'hi12345');

Function handles each column separately , The main processing logic takes three branches ：

3.3.1 The entry logic of three branches

Branch one ： atti->attlen == -1 And atti->attstorage != 'p' And Now it's 4B head And The data is very short and can be changed into 1B head
Branch two ： atti->attlen == -1 And Now it's 1B_E head And 1B_E yes RO type VARTAG_EXPANDED_RO
Branch three ： Other situations 

		if (ATT_IS_PACKABLE(atti) &&
			VARATT_CAN_MAKE_SHORT(DatumGetPointer(val)))
		{
			/*
			 * we're anticipating converting to a short varlena header, so
			 * adjust length and don't count any alignment
			 */
			data_length += VARATT_CONVERTED_SHORT_SIZE(DatumGetPointer(val));
		}
		else if (atti->attlen == -1 &&
				 VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(val)))
		{
			/*
			 * we want to flatten the expanded value so that the constructed
			 * tuple doesn't depend on it
			 */
			data_length = att_align_nominal(data_length, atti->attalign);
			data_length += EOH_get_flat_size(DatumGetEOHP(val));
		}
		else
		{
			data_length = att_align_datum(data_length, atti->attalign,
										  atti->attlen, val);
			data_length = att_addlength_datum(data_length, atti->attlen,
											  val);
		}

For five columns of test data

int type ： Take branch three （ length 4）

(gdb) p atti->attlen
$30 = 4
(gdb) p atti->attstorage
$31 = 112 'p'

Calculation process 

//  First step ： alignment data_length=0, After alignment, it is still 0
			data_length = att_align_datum(data_length, atti->attalign,
										  atti->attlen, val);
//  The second step ： Plus length atti->attlen,data_length=4
			data_length = att_addlength_datum(data_length, atti->attlen,
											  val);

Increase in length 4

varchar type ： Take a branch （ length 8）

(gdb) p atti->attlen
$38 = -1
(gdb) p atti->attstorage
$39 = 120 'x'

Calculation process 

//  can 1B It can be installed ,  Behind the 4B Turn into 1B head , Here by 1B Calculate the length 
data_length += VARATT_CONVERTED_SHORT_SIZE(DatumGetPointer(val))

Increase in length 8

numeric type ： Take a branch （ length 7）

char type ： Take a branch （ length 3）

1B Add yourself to your head 2 Bytes , Three bytes in total

text type ： Take a branch （ length 8）

1B Add yourself to your head 7 Bytes , altogether 8 byte

3.4 heap_fill_tuple

heap_fill_tuple Call... For each column fill_val Fill in data 

heap_fill_tuple
  for (i = 0; i < numberOfAttributes; i++)
    fill_val(...)

fill_val There are more branches of , For each column, go to the following 4 Branches to handle 

    if (att->attbyval)
	{
		/* pass-by-value */
		data = (char *) att_align_nominal(data, att->attalign);
		store_att_byval(data, datum, att->attlen);
		data_length = att->attlen;
	}
	else if (att->attlen == -1)
	{
		/* varlena */
		Pointer		val = DatumGetPointer(datum);

		*infomask |= HEAP_HASVARWIDTH;
		if (VARATT_IS_EXTERNAL(val))
		{
			if (VARATT_IS_EXTERNAL_EXPANDED(val))
			{
				/*
				 * we want to flatten the expanded value so that the
				 * constructed tuple doesn't depend on it
				 */
				ExpandedObjectHeader *eoh = DatumGetEOHP(datum);

				data = (char *) att_align_nominal(data,
												  att->attalign);
				data_length = EOH_get_flat_size(eoh);
				EOH_flatten_into(eoh, data, data_length);
			}
			else
			{
				*infomask |= HEAP_HASEXTERNAL;
				/* no alignment, since it's short by definition */
				data_length = VARSIZE_EXTERNAL(val);
				memcpy(data, val, data_length);
			}
		}
		else if (VARATT_IS_SHORT(val))
		{
			/* no alignment for short varlenas */
			data_length = VARSIZE_SHORT(val);
			memcpy(data, val, data_length);
		}
		else if (VARLENA_ATT_IS_PACKABLE(att) &&
				 VARATT_CAN_MAKE_SHORT(val))
		{
			/* convert to short varlena -- no alignment */
			data_length = VARATT_CONVERTED_SHORT_SIZE(val);
			SET_VARSIZE_SHORT(data, data_length);
			memcpy(data + 1, VARDATA(val), data_length - 1);
		}
		else
		{
			/* full 4-byte header varlena */
			data = (char *) att_align_nominal(data,
											  att->attalign);
			data_length = VARSIZE(val);
			memcpy(data, val, data_length);
		}
	}
	else if (att->attlen == -2)
	{
		/* cstring ... never needs alignment */
		*infomask |= HEAP_HASVARWIDTH;
		Assert(att->attalign == TYPALIGN_CHAR);
		data_length = strlen(DatumGetCString(datum)) + 1;
		memcpy(data, DatumGetPointer(datum), data_length);
	}
	else
	{
		/* fixed-length pass-by-reference */
		data = (char *) att_align_nominal(data, att->attalign);
		Assert(att->attlen > 0);
		data_length = att->attlen;
		memcpy(data, DatumGetPointer(datum), data_length);
	}

Branch ：

  1. att->attbyval == true Values are passed directly , Just assign values directly
  2. att->attlen == -1 Variable length head type , To go valena Press 4B、1B、1B_E Deal with... Separately
  3. att->attlen == -2 Direct copy cstring type
  4. other ： Direct copy

For five columns of test data

int type ： Take a branch ： Value copy

The data transferred is stored in the stack memory , Direct assignment

varchar type ： Take branch two ： data 4B Convert to 1B Post memory copy

The data is small enough , Don't have to 4B Header storage , Convert to 1B Copy the header after saving

numeric type ： Take branch two ： data 4B Convert to 1B Post memory copy

The data is small enough , Don't have to 4B Header storage , Convert to 1B Copy the header after saving

char type ： Take branch two ： data 4B Convert to 1B Post memory copy

The data is small enough , Don't have to 4B Header storage , Convert to 1B Copy the header after saving

text type ： Take branch two ： data 4B Convert to 1B Post memory copy

The data is small enough , Don't have to 4B Header storage , Convert to 1B Copy the header after saving

原网站

版权声明
本文为[mingjie73]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/182/202207012257394643.html

边栏推荐

猜你喜欢

随机推荐