I'm not sure if we are suffering from similar problem as I do think m1 share similar architecture to AMD processors in terms of CPU cache architectures and lack of AVX512 support. The closest I can get to is to force pt_to_tf.py to load the whole model then save it, however an SIGKILL (most likely OOM kill) kicked in.ĭo you have, by any chance, an idea how to properly convert the bin file back to tensorflow ckpt layer by layer on the fly like what you did on loading then inferencing Sorry if I deviated from the original topic as well. ![]() I tried to mildly modify both transformers/convert_pytorch_checkpoint_to_tf2.py and transformers/commands/pt_to_tf.py and take a chance either of them would work. ![]() I'm trying to mitigate this by adopting tensorflow-metal (by Apple) which presumably be more polished on macOS, however couldn't find an easy way to convert bloom pt back to tensorflow checkpoints. ![]() Hi I'm trying to infer bloom on my apple silicon Mac (20c 128G), however model runs extremely slow on CPU (60s/layer, seemingly not properly parallelized) nor mps backend working properly (outputs identical token for various inputs, 0.1s/layer though).
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |