chore: move rai_grounding_dino to rai_extensions #195
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maciejmajek
commented
Sep 4, 2024
src/rai_extensions/rai_grounding_dino/rai_grounding_dino/grounding_dino.py
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
We are planning a lot of extensions. Placing them all directly in the src/ folder could be cluttering and make the project structure less organized.
Proposed Changes
This PR adds a new
rai_extensionsfolder and moves the existingrai_grounding_dinoextension into it. Specifically:rai_extensionsfolder in thesrcdirectoryrai_grounding_dinofolder intorai_extensionsTesting
Update:
I've reinstalled my poetry environment and now it works properly.
Run log ✅
[INFO] [launch]: All log files can be found below /home/mmajek/.ros/log/2024-09-04-18-09-46-942751-robo-pc-054-1638270
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [grounding_dino-1]: process started with pid [1638271]
[INFO] [talker-2]: process started with pid [1638273]
[grounding_dino-1] UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
[talker-2] [INFO] [1725466188.292966869] [example_client]: service not available, waiting again...
[grounding_dino-1] UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3587.)
[grounding_dino-1] FutureWarning:
clean_up_tokenization_spaceswas not set. It will be set toTrueby default. This behavior will be depracted in transformers v4.45, and will be then set toFalseby default. For more details check this issue: huggingface/transformers#31884[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed inte 8000 rnally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
betawill be renamed internally tobias. Please use a different name to suppress this warning.[grounding_dino-1] A parameter name that contains
gammawill be renamed internally toweight. Please use a different name to suppress this warning.[grounding_dino-1] [INFO] [1725466190.171876145] [grounding_dino]: Request received: dragon , lizard , dinosaur, 0.4, 0.4
[grounding_dino-1] FutureWarning: The
deviceargument is deprecated and will be removed in v5 of Transformers.[grounding_dino-1] UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.
[grounding_dino-1] UserWarning: None of the inputs have requires_grad=True. Gradients will be None
[talker-2] [INFO] [1725466191.535909371] [example_client]: rai_interfaces.msg.RAIDetectionArray(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=1725466191, nanosec=534060977), frame_id=''), detections=[vision_msgs.msg.Detection2D(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=1725466191, nanosec=533842724), frame_id=''), results=[vision_msgs.msg.ObjectHypothesisWithPose(hypothesis=vision_msgs.msg.ObjectHypothesis(class_id='dinosaur', score=0.4548650085926056), pose=geometry_msgs.msg.PoseWithCovariance(pose=geometry_msgs.msg.Pose(position=geometry_msgs.msg.Point(x=0.0, y=0.0, z=0.0), orientation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=0.0, w=1.0)), covariance=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0.])))], bbox=vision_msgs.msg.BoundingBox2D(center=vision_msgs.msg.Pose2D(position=vision_msgs.msg.Point2D(x=1170.7906494140625, y=521.2423095703125), theta=0.0), size_x=578.667236328125, size_y=315.9845886230469), id=''), vision_msgs.msg.Detection2D(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=1725466191, nanosec=533965717), frame_id=''), results=[vision_msgs.msg.ObjectHypothesisWithPose(hypothesis=vision_msgs.msg.ObjectHypothesis(class_id='dinosaur', score=0.46678999066352844), pose=geometry_msgs.msg.PoseWithCovariance(pose=geometry_msgs.msg.Pose(position=geometry_msgs.msg.Point(x=0.0, y=0.0, z=0.0), orientation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=0.0, w=1.0)), covariance=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0.])))], bbox=vision_msgs.msg.BoundingBox2D(center=vision_msgs.msg.Pose2D(position=vision_msgs.msg.Point2D(x=403.49383544921875, y=459.339599609375), theta=0.0), size_x=531.4512939453125, size_y=428.9387512207031), id=''), vision_msgs.msg.Detection2D(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=1725466191, nanosec=533996285), frame_id=''), results=[vision_msgs.msg.ObjectHypothesisWithPose(hypothesis=vision_msgs.msg.ObjectHypothesis(class_id='dinosaur', score=0.4597964882850647), pose=geometry_msgs.msg.PoseWithCovariance(pose=geometry_msgs.msg.Pose(position=geometry_msgs.msg.Point(x=0.0, y=0.0, z=0.0), orientation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=0.0, w=1.0)), covariance=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0.])))], bbox=vision_msgs.msg.BoundingBox2D(center=vision_msgs.msg.Pose2D(position=vision_msgs.msg.Point2D(x=1418.049072265625, y=430.03955078125), theta=0.0), size_x=552.552490234375, size_y=415.8872375488281), id=''), vision_msgs.msg.Detection2D(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=1725466191, nanosec=534025510), frame_id=''), results=[vision_msgs.msg.ObjectHypothesisWithPose(hypothesis=vision_msgs.msg.ObjectHypothesis(class_id='dragon', score=0.40476351976394653), pose=geometry_msgs.msg.PoseWithCovariance(pose=geometry_msgs.msg.Pose(position=geometry_msgs.msg.Point(x=0.0, y=0.0, z=0.0), orientation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=0.0, w=1.0)), covariance=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
[talker-2] 0., 0.])))], bbox=vision_msgs.msg.BoundingBox2D(center=vision_msgs.msg.Pose2D(position=vision_msgs.msg.Point2D(x=403.4332275390625, y=459.6795654296875), theta=0.0), size_x=531.529541015625, size_y=429.6009521484375), id='')], detection_classes=['dragon', 'lizard', 'dinosaur'])
Without
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=pythonthere is an error