如何修改线程内的列表?

时间:2021-02-11 20:58:55

I need to modify a list by adding some elements at the end from a thread.

我需要通过在线程末尾添加一些元素来修改列表。

This is my code:

这是我的代码:

def go():
  while queueCommand.qsize() > 0:
      command = queueCommand.get(False)
      res = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
      output = res.communicate()
      output = str(output)
      star = output.find("address") + 8
      en = output.find(",")
      ip = output[star:en-3]  
      alist.append(ip)  #<================== her i use the liste

if __name__ == '__main__':      
  with open ("icq.txt","r") as myfile:
    text = myfile.read()
  end = 0
  alist = []
  queueCommand = Queue.Queue()
  while True:
    start = text.find("href=") + 13
    if start == 12:
      break
    end = text.find("/",start)
    if text[start:end].find("icq.com") != -1:
      hostname="host "+ text[start:end]
      queueCommand.put(hostname) 
    text = text[end:len(text)]  

  for i in range(10):   
    threading.Thread(target=go,args=(alist)).start() #<====== i give the list as argument

  print alist 

The last print statement display an empty list, []. Any ideas?

最后一个print语句显示一个空列表[]。有任何想法吗?

1 个解决方案

#1


1  

You have a few problems.

你有一些问题。

  1. You specify alist as the args, but you need to pass it as a tuple which it looks like you tried to do, but a one item tuple would look like this (alist,). Right now you are just using the alist global which is likely not what you want.

    你将alist指定为args,但是你需要将它作为一个看起来像你试图做的元组传递,但是一个项目元组看起来像这样(alist,)。现在你只是使用alist global,这可能不是你想要的。

  2. Your go method does not expect an argument (namely, the alist).

    你的go方法不期望一个参数(即alist)。

  3. To be thread safe, I believe you will need to use some sort of semaphore/mutex/lock primitive. The threading module comes with a Lock implementation which you can use to restrict access to alist during the append operation.

    为了线程安全,我相信你需要使用某种信号量/互斥/锁定原语。线程模块附带一个Lock实现,您可以使用该实现在追加操作期间限制对alist的访问。

  4. Most importantly, you are not waiting for your threads to finish before you print the result. To wait for thread to finish you need to call .join() on the thread.

    最重要的是,在打印结果之前,您不会等待线程完成。要等待线程完成,您需要在线程上调用.join()。

I would probably opt for using another Queue instance to put the results into, then you could read all from the queue to construct your list once the threads are complete.

我可能会选择使用另一个Queue实例将结果放入,然后您可以从队列中读取所有内容以在线程完成后构建列表。

Here is an updated version (working) of your code. Like I said, I would probably opt to use a Queue instead, and I have not used the threading module much since I switched to eventlet/gevent... so there may be ways to improve on what I provided.

这是您的代码的更新版本(工作)。就像我说的那样,我可能会选择使用Queue,而且我没有使用过线程模块,因为我切换到了eventlet / gevent ......因此可能有办法改进我提供的内容。

import threading
import Queue
import subprocess

lock = threading.Lock()

def go(alist):
  while queueCommand.qsize() > 0:
      command = queueCommand.get(False)
      res = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
      output = res.communicate()
      output = str(output)
      star = output.find("address") + 8
      en = output.find(",")
      ip = output[star:en-3]
      lock.acquire()
      alist.append(ip)  #<================== her i use the liste
      lock.release()

def foo(alist):
    alist.append("bar")

if __name__ == '__main__':
  with open ("icq.txt","r") as myfile:
    text = myfile.read()
  end = 0
  alist = []
  queueCommand = Queue.Queue()
  while True:
    start = text.find("href=") + 13
    if start == 12:
      break
    end = text.find("/",start)
    if text[start:end].find("icq.com") != -1:
      hostname="host "+ text[start:end]
      queueCommand.put(hostname)
    text = text[end:len(text)]

  threads = []
  for i in range(10):
    thread = threading.Thread(target=go,args=(alist,)) #<====== i give the list as argument)
    thread.start()
    threads.append(thread)
  for thread in threads:
    thread.join()

  print alist

#1


1  

You have a few problems.

你有一些问题。

  1. You specify alist as the args, but you need to pass it as a tuple which it looks like you tried to do, but a one item tuple would look like this (alist,). Right now you are just using the alist global which is likely not what you want.

    你将alist指定为args,但是你需要将它作为一个看起来像你试图做的元组传递,但是一个项目元组看起来像这样(alist,)。现在你只是使用alist global,这可能不是你想要的。

  2. Your go method does not expect an argument (namely, the alist).

    你的go方法不期望一个参数(即alist)。

  3. To be thread safe, I believe you will need to use some sort of semaphore/mutex/lock primitive. The threading module comes with a Lock implementation which you can use to restrict access to alist during the append operation.

    为了线程安全,我相信你需要使用某种信号量/互斥/锁定原语。线程模块附带一个Lock实现,您可以使用该实现在追加操作期间限制对alist的访问。

  4. Most importantly, you are not waiting for your threads to finish before you print the result. To wait for thread to finish you need to call .join() on the thread.

    最重要的是,在打印结果之前,您不会等待线程完成。要等待线程完成,您需要在线程上调用.join()。

I would probably opt for using another Queue instance to put the results into, then you could read all from the queue to construct your list once the threads are complete.

我可能会选择使用另一个Queue实例将结果放入,然后您可以从队列中读取所有内容以在线程完成后构建列表。

Here is an updated version (working) of your code. Like I said, I would probably opt to use a Queue instead, and I have not used the threading module much since I switched to eventlet/gevent... so there may be ways to improve on what I provided.

这是您的代码的更新版本(工作)。就像我说的那样,我可能会选择使用Queue,而且我没有使用过线程模块,因为我切换到了eventlet / gevent ......因此可能有办法改进我提供的内容。

import threading
import Queue
import subprocess

lock = threading.Lock()

def go(alist):
  while queueCommand.qsize() > 0:
      command = queueCommand.get(False)
      res = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
      output = res.communicate()
      output = str(output)
      star = output.find("address") + 8
      en = output.find(",")
      ip = output[star:en-3]
      lock.acquire()
      alist.append(ip)  #<================== her i use the liste
      lock.release()

def foo(alist):
    alist.append("bar")

if __name__ == '__main__':
  with open ("icq.txt","r") as myfile:
    text = myfile.read()
  end = 0
  alist = []
  queueCommand = Queue.Queue()
  while True:
    start = text.find("href=") + 13
    if start == 12:
      break
    end = text.find("/",start)
    if text[start:end].find("icq.com") != -1:
      hostname="host "+ text[start:end]
      queueCommand.put(hostname)
    text = text[end:len(text)]

  threads = []
  for i in range(10):
    thread = threading.Thread(target=go,args=(alist,)) #<====== i give the list as argument)
    thread.start()
    threads.append(thread)
  for thread in threads:
    thread.join()

  print alist