标签云

微信群

扫码加入我们

WeChat QR Code

With the tornado I use multiprocessing to create a child process to do the compressing work. And the child process directly send data to main process via os.pipeRight, now the problem is self.stream.read_until_close block the call, and it does not tar_worker close the other side of the pipe.My code list below:def tar_worker(fd, path):'''Tar worker to do the compressing work, directly write data to pipe.'''fp = os.fdopen(fd, 'w')tar = tarfile.open(mode='w|', fileobj=fp, dereference=True)tar.add(path)tar.close()fp.close()class getTarFileHandler(tornado.web.RequestHandler):@tornado.web.asynchronous@gen.coroutinedef get(self):recv, sender = os.pipe()self.p = multiprocessing.Process(target=tar_worker, args=(sender, '/boot/'))self.p.start()self.fd = recv# Create PipeIOStreamself.stream = tornado.iostream.PipeIOStream(self.fd)self.stream.read_until_close(callback=self.f, streaming_callback=self.send_data)def f(self, s):print 'done send work'self.finish() #close connectiondef send_data(self, chunk):self.write(chunk)self.flush()


You use gen.coroutine, but then use callbacks instead of yield. You need to remove the coroutine decorator or replace the callbacks with yield. There is also a typo: you set self.fd = recv but use self.fp when creating the PipeIOStream.

2019年04月22日43分07秒

I replace the callback to yield self.stream.read_until_close(callback=self.f, streaming_callback=self.send_data) and add self.f() under this line, but still not close the connection.

2019年04月22日43分07秒

Don't use a callback= argument when using yield (but you do want to use a callback for streaming_callback): yield self.stream.read_until_close(streaming_callback=self.send_data). It's often more idiomatic in coroutines to use a loop with data = yield self.stream.read_bytes(chunk_size, partial=True).

2019年04月22日43分07秒

I found this bug maybe not relate to tornado, is a os pipe problem? I do r, w=os.pipe() and set r to unblocked, and close w, then os.read(r, 10) return '' instead of raise errno.EBADF

2019年04月22日43分07秒

You should not expect EBADF in this case. The read and write ends of a pipe are closed independently, so r is still open even after w is closed. read() returns an empty string when you have reached the end of the pipe.

2019年04月22日43分07秒