You can control the rate of processed requests by creating a queue. This will allow you to set the number of requests to be processed at one time, and will let users know their position in the queue. Parameters: concurrency_count: Number of worker threads that will be pr
(
self,
concurrency_count: int = 1,
status_update_rate: float | str = "auto",
client_position_to_load_data: int = 30,
default_enabled: bool = True,
api_open: bool = True,
max_size: Optional[int] = None,
)
| 1133 | |
| 1134 | @document() |
| 1135 | def queue( |
| 1136 | self, |
| 1137 | concurrency_count: int = 1, |
| 1138 | status_update_rate: float | str = "auto", |
| 1139 | client_position_to_load_data: int = 30, |
| 1140 | default_enabled: bool = True, |
| 1141 | api_open: bool = True, |
| 1142 | max_size: Optional[int] = None, |
| 1143 | ): |
| 1144 | """ |
| 1145 | You can control the rate of processed requests by creating a queue. This will allow you to set the number of requests to be processed at one time, and will let users know their position in the queue. |
| 1146 | Parameters: |
| 1147 | concurrency_count: Number of worker threads that will be processing requests concurrently. |
| 1148 | status_update_rate: If "auto", Queue will send status estimations to all clients whenever a job is finished. Otherwise Queue will send status at regular intervals set by this parameter as the number of seconds. |
| 1149 | client_position_to_load_data: Once a client's position in Queue is less that this value, the Queue will collect the input data from the client. You may make this smaller if clients can send large volumes of data, such as video, since the queued data is stored in memory. |
| 1150 | default_enabled: If True, all event listeners will use queueing by default. |
| 1151 | api_open: If True, the REST routes of the backend will be open, allowing requests made directly to those endpoints to skip the queue. |
| 1152 | max_size: The maximum number of events the queue will store at any given moment. |
| 1153 | Example: |
| 1154 | demo = gr.Interface(gr.Textbox(), gr.Image(), image_generator) |
| 1155 | demo.queue(concurrency_count=3) |
| 1156 | demo.launch() |
| 1157 | """ |
| 1158 | self.enable_queue = default_enabled |
| 1159 | self.api_open = api_open |
| 1160 | self._queue = queue.Queue( |
| 1161 | live_updates=status_update_rate == "auto", |
| 1162 | concurrency_count=concurrency_count, |
| 1163 | data_gathering_start=client_position_to_load_data, |
| 1164 | update_intervals=status_update_rate if status_update_rate != "auto" else 1, |
| 1165 | max_size=max_size, |
| 1166 | blocks_dependencies=self.dependencies, |
| 1167 | ) |
| 1168 | self.config = self.get_config_file() |
| 1169 | return self |
| 1170 | |
| 1171 | def launch( |
| 1172 | self, |