在kdb/q保存熊猫数据。

What is the best way to save a pandas dataframe in kdb? Are there any libraries that can make it easier?

在kdb保存熊猫数据档案最好的方法是什么?有什么库可以使它更简单吗?

The below code can apparently be used to loas something from kdb, but how do I save a dataframe into it?

下面的代码显然可以从kdb中使用loas，但是如何将dataframe保存到它中呢?

from qpython import qconnection

with qconnection.QConnection(host = 'localhost', port = 5001, pandas = True) as q:
    ds = q('(1i;0Ni;3i)', pandas = True)
    print(ds)

3 个解决方案

#1

To save a dataframe in KDB+ with qPython one can use the sync method of the QConnection class. Set the first parameter to a string defining a q function that assigns its parameter to a global variable and send the dataframe as the second parameter. Something like this:

要使用qPython在KDB+中保存dataframe，可以使用QConnection类的同步方法。将第一个参数设置为定义q函数的字符串，该函数将其参数分配给全局变量并将dataframe作为第二个参数。是这样的:

from qpython import qconnection
import pandas as pd
df = pd.DataFrame({'sym':['abc','def','ghi'],'price':[10.1,10.2,10.3]})
with qconnection.QConnection(host = 'localhost', port = 5001, pandas = True) as q:
    q.sync('{t::x}',df)

Note that you need to use a double colon :: in the function definition so that the parameter is assigned to a global variable t rather that a local variable.

注意，需要在函数定义中使用双冒号::，以便将参数分配给全局变量t，而不是局部变量t。

#2

Using a mock table from Pandas Integration

使用熊猫集成的模拟表

with qconnection.QConnection(host = 'localhost', port = 5000, pandas = True) as q:

 df =  q('flip `name`iq`fullname!(`Dent`Beeblebrox`Prefect;98 42 126;("Arthur Dent";"Zaphod Beeblebrox"; "Ford Prefect"))')

This pulls a table built in kdb back in python as a pandas dataframe. The following method saves the table down in memory:

这将把用kdb构建的作为熊猫dataframe的表拉回来。以下方法将该表保存在内存中:

with qconnection.QConnection(host = 'localhost', port = 5000, pandas = True) as q:

  q('{`t set x}', df)

Some more information on how data is saved in kdb can be found: kdb overview

关于如何在kdb中保存数据的更多信息可以找到:kdb概述

More examples of using set to save data can be found here: set

在这里可以找到更多使用set来保存数据的例子:set。

There is another method of integrating python and q that may be of use to you; PyQ brings the Python and q interpreters into the same process, so that code written in either of the languages operates on the same data.

还有一种集成python和q的方法可能对您有用;PyQ将Python和q解释器引入到同一个进程中，因此用这两种语言编写的代码在相同的数据上运行。

#3

One option is to use embedPy which allows kdb+ and Python to share the same process and memory space

一种选择是使用embedPy，允许kdb+和Python共享相同的进程和内存空间

You can find documation on this library in the provide link http://code.kx.com/q/ml/embedpy/

您可以在提供链接http://code.kx.com/q/ml/embedpy/中找到该库的文档

See below an example implemented in kdb+

请参见下面在kdb+中实现的示例

    q)/ Create kdb+ table
    q)n:100;show 5#qtab:([]scol:n?`AAA`BBB`CCC;icol:n?100;fcol:n?1f)
       scol icol vcol     
      -------------------
       CCC  59   0.3927524
       AAA  30   0.5170911
       CCC  85   0.5159796
       AAA  89   0.4066642
       AAA  23   0.1780839
    q)
    q)/ Load embedPy and import pandas/DataFrame
    q)\l p.q
    q)df:(pd:.p.import`pandas)`:DataFrame
    q)
    q)/ Initialize DataFrame object
    q)/ need to first covert qtable to dictionary 
    q)/ use 'from_dict' to  change it to python dataframe 
    q)ptab:df[`:from_dict]flip qtab
    q)/ print is built in embedPy
    q)print ptab[`:head]5;
      icol scol      vcol
      -------------------
       0    59  CCC  0.392752
       1    30  AAA  0.517091
       2    85  CCC  0.515980
       3    89  AAA  0.406664
       4    23  AAA  0.178084
    q)/ and converting back to kdb+
    q)/ need to convert it to a dict like 
    q)5#flip ptab[`:to_dict;`list]`
      icol scol  vcol     
     --------------------
      59   "CCC" 0.3927524
      30   "AAA" 0.5170911
      85   "CCC" 0.5159796
      89   "AAA" 0.4066642
      23   "AAA" 0.1780839
   q)
   q)/ can also use but slower
   q)5#ptab[`:to_dict;`records]`
     icol scol  vcol     
    --------------------
      59   "CCC" 0.3927524
      30   "AAA" 0.5170911
      85   "CCC" 0.5159796
      89   "AAA" 0.4066642
      23   "AAA" 0.1780839
  q)
  q)/ Consider keyed table
  q)show qktab:select by scol from qtab
    scol| icol vcol      
    ----| ---------------
    AAA | 35   0.3410485 
    BBB | 61   0.5548864 
    CCC | 0    0.07347808
  q)/ print as dataframe format
  q)print pktab:df[`:from_dict;flip 0!qktab]
    icol scol      vcol
0    35  AAA  0.341049
1    61  BBB  0.554886
2     0  CCC  0.073478
  q) / convert it to a dataframe keyed table
  q) / setting the index to become the keyed column
  q) / parameter requires to be a python list
  q)
  q)print pktab:pktab[`:set_index].p.eval["list"]keys qktab
          icol      vcol
  scol                
  AAA       35  0.341049
  BBB       61  0.554886
  CCC      0  0.073478
 q) / converting back to kdb+
 q) / need to `reset_index` to return full table
 q) / then key the table 
 q)(`$pktab[`:index.names]`)xkey flip pktab[`:reset_index][][`:to_dict;`list]`
  scol | icol vcol      
  -----| ---------------
  "AAA"| 35   0.3410485 
  "BBB"| 61   0.5548864 
  "CCC"| 0    0.07347808
q)

#1

from qpython import qconnection
import pandas as pd
df = pd.DataFrame({'sym':['abc','def','ghi'],'price':[10.1,10.2,10.3]})
with qconnection.QConnection(host = 'localhost', port = 5001, pandas = True) as q:
    q.sync('{t::x}',df)